About
Posts
Projects
Tags

Connectivity Tests in GCP: How to Verify and Troubleshoot Your VPC Networks

GCP Networking Guide
Updated on
Connectivity Tests in GCP: How to Verify and Troubleshoot Your VPC Networks

Introduction

When you spin up Virtual Private Cloud (VPC) networks, firewall rules, Cloud VPNs, or Private Service Connect in Google Cloud Platform, you need confidence that your services can actually talk to each other. Connectivity issues often hide in subtle misconfigurations: a stray firewall tag, an overlooked route, or a peering hiccup. Rather than manually SSH-ing into VMs and running traceroutes by hand, GCP’s Connectivity Tests (part of Network Intelligence Center) lets you define, run, and automate reachability checks between any two endpoints.

In this post you’ll learn:

  • What Connectivity Tests are and why they matter
  • How to set up and run tests via the gcloud CLI
  • How to interpret results and automate checks
  • Best practices to integrate connectivity tests into your workflow

Background & Problem Statement

Even in well-architected networks, changes to firewall rules, new subnets, or added VPN tunnels can inadvertently block traffic. Traditional debugging—SSH into source VM, run ping or traceroute, tweak a rule, repeat—doesn’t scale as you manage dozens of services across multiple regions.

Connectivity Tests solves this by:

  1. Modeling your network topology automatically
  2. Simulating packet flows between chosen endpoints
  3. Reporting exactly where packets would be dropped or accepted

This lets you catch issues before they impact customers.

Deep Dive / Main Sections

Setting Up a Connectivity Test

Explanation A Connectivity Test is a GCP resource you define once; afterwards you can run it on-demand or schedule it.

Example

# 1. Create a test named "vm-to-db"
gcloud network-management connectivity-tests create vm-to-db \
  --source="projects/my-proj/zones/us-central1-a/instances/web-vm" \
  --destination="projects/my-proj/zones/us-central1-b/instances/db-vm" \
  --protocol=TCP \
  --destination-port=5432

Key Takeaways

  • --source and --destination accept VMs, IPs, or GKE services
  • Default protocol is TCP; you can specify UDP or ICMP
  • Port helps pinpoint firewall or route issues

Running and Interpreting Results

Explanation Once created, a test definition persists in your project. Running it simulates connectivity and surfaces the precise hop where packets fail.

Example

# Run the test synchronously
gcloud network-management connectivity-tests run vm-to-db

# View detailed results
gcloud network-management connectivity-tests describe vm-to-db \
  --format="yaml"

A successful test returns:

results:
  verdict: "PASS"
  traces:
    - steps:
        - index: 1
          state: FORWARDED
          ...

If it fails, you’ll see a DROP and the step will indicate whether a firewall or route blocked the packet.

Key Takeaways

  • Look for verdict: PASS vs. DROP
  • The state field pinpoints the failure reason (e.g., DENIED_BY_FIREWALL)
  • Use --format="json" or yaml for automation

Automating Connectivity Checks

Explanation Integrate connectivity tests into CI/CD pipelines or scheduled checks via Cloud Scheduler + Cloud Functions to proactively alert teams when network paths break.

Example (CI pipeline snippet)

# .gitlab-ci.yml example
connectivity_test:
  image: google/cloud-sdk:latest
  stage: test
  script:
    - gcloud auth activate-service-account --key-file=${GCP_SA_KEY}
    - gcloud network-management connectivity-tests run vm-to-db --format=json > result.json
    - |
      if grep -q '"verdict": "FAIL"' result.json; then
        echo "Connectivity test failed"; exit 1;
      fi

Key Takeaways

  • Fail your deployment if critical paths are broken
  • Store test definitions in IaC (Terraform, Deployment Manager) for versioning
  • Combine with alerting (e.g., send Slack notification on failures)

Visualizing with the Console

Explanation While CLI is great for automation, the GCP Console offers a graphical Topology view showing your test’s packet path through routers, firewalls, and VPNs.

  1. Navigate to Network Intelligence Center → Connectivity Tests
  2. Select your test and click View details
  3. Examine the topology map and step-by-step trace

Key Takeaways

  • Visual map accelerates root-cause analysis
  • Handy for onboarding teammates unfamiliar with gcloud CLI

Best Practices / Tips & Tricks

  1. Define tests for every critical path (frontend → API, API → DB, VPN → On-prem)
  2. Version-control your test definitions alongside Terraform or Deployment Manager
  3. Schedule daily or hourly runs for production networks, less frequently for dev/test
  4. Alert on change of verdict: integrate with Cloud Monitoring uptime checks
  5. Label your tests (env=prod, team=payments) to filter large inventories

Common Pitfalls & FAQs

IssueSolution
“Test definition already exists”Use gcloud network-management connectivity-tests update
Missing IAM permissionGrant networkmanagement.connectivityTests.run to your service account
Cross-project source/destEnsure both projects share VPC or enable Shared VPC peering
Test stuck in RUNNING stateAdd --timeout flag (e.g., --timeout=30s)

Conclusion

Connectivity Tests in GCP empower you to shift network debugging left—catching misconfigurations before they hit production. By defining, automating, and visualizing reachability checks, you gain both confidence and clarity over your VPC landscape.

Call to Action

References & Further Reading