Zero to Multi-Region: High Availability Serverless with Cloud Run and Cross-Region Failover & Failback
DEV Community

Zero to Multi-Region: High Availability Serverless with Cloud Run and Cross-Region Failover & Failback

Google just made multi-region Cloud Run significantly easier. Here is the full picture: what changed, what it means in practice, and how to build it right.

Most teams discover they need multi-region architecture the hard way and, sadly, during an outage. Whether you're running a global e-commerce platform, a real-time gaming API, or a financial services application, users expect your service to be available whenever they need it. There is a conversation that happens in almost every engineering team at some point. It usually starts with a post-mortem. A regional Google Cloud outage or a Cloud Run service that hit a cold start spike, or a single-region deployment that could not handle the latency demands of users spread across Lagos, Nairobi, and London simultaneously, caused enough pain that someone finally asked: why are we only deployed in one region? The answer is usually one of three things: it felt complex, it felt expensive, or no one had prioritised it yet.

In July 2026, Google moved Cloud Run Service Health to General Availability and the timing was hard to miss. Six days earlier, a power cut at Google's Netherlands data centre had knocked three services offline. The GA release brings automatic cross-region failover to Cloud Run with what Google describes as a two-step setup: add a readiness probe, set minimum instances to at least 1. The load balancer does the rest. This article covers the full architecture, what Service Health is, how readiness probes underpin it, how to set up the Global Load Balancer correctly, and how to test that failover actually works. It also covers the production details.

What changed: Service Health and readiness probes

Before Service Health, multi-region Cloud Run required you to implement a /health endpoint in your application and configure a separate HTTPS health check at the load balancer level. This worked, but it had a significant gap. The load balancer's health check only knew whether the Cloud Run service endpoint was responding, not whether the individual container instances behind it were actually ready to serve traffic.

Service Health introduces two new capabilities that close this gap:

  • Readiness probes operate at the container instance level. Cloud Run periodically sends an HTTP request to a path you specify on each running container instance. If the probe fails, Cloud Run stops routing requests to that instance until the probe succeeds again. Critically, a failing readiness probe does not kill the instance (that is what a liveness probe does), it simply marks the instance as not ready for traffic.
  • Service Health aggregates the readiness state of all container instances in a region into a single regional health signal. This aggregated health status is exposed through the Serverless NEGs (Network Endpoint Groups) for that region. When the Global Load Balancer reads the NEG's health status and sees a region is unhealthy, because enough instances are failing their readiness probes; it automatically reroutes traffic to a healthy region. When the failing region recovers, traffic is gradually restored without any operator action.

The result: failover and failback capabilities are now fully automated, triggered by real instance-level health rather than a synthetic endpoint check.

Container instance (readiness probe fails)
       │
       ▼
Cloud Run aggregates probe results across all instances in the region
to determine the overall health status of each regional service
       │
       ▼
Service Health: region marked UNHEALTHY
       │
       ▼
Serverless NEG reports unhealthy status to Global Load Balancer
       │
       ▼
Load Balancer stops routing to this region → shifts traffic to healthy region
       │
       ▼
Region recovers → Load Balancer gradually restores traffic

This is available in all Cloud Run regions at no extra charge beyond the CPU and memory consumed while readiness probes run.

Architecture overview

┌──────────────────────────────────┐
│      Global Anycast IP (single IP)│
│      + SSL Certificate (managed)    │
└───────────────┬──────────────────┘
                │
┌───────────────▼──────────────────┐
│   Global External HTTP(S) LB     │
│   (URL map + forwarding rules)   │
└──────┬──────────────────┬────────┘
       │                  │
┌──────────────────▼──┐ ┌─────▼──────────────────┐
│  Serverless NEG     │ │  Serverless NEG         │
│  africa-south1      │ │  us-central1            │
│  (Service Health    │ │  (Service Health        │
│   status: healthy)  │ │   status: healthy)      │
└──────────┬──────────┘ └───────────┬────────────┘
           │                        │
┌───────────────▼───────────┐ ┌──────────────▼──────────────┐
│  Cloud Run Service        │ │  Cloud Run Service           │
│  africa-south1            │ │  us-central1                 │
│  Readiness probe: /health │ │  Readiness probe: /health   │
│  min-instances: 1+        │ │  min-instances: 1+          │
│  (auto-scales 0-N)        │ │  (auto-scales 0-N)          │
└───────────────────────────┘ └─────────────────────────────┘

Prerequisites and setup

export PROJECT_ID = "your-project-id"
export PROJECT_NUMBER = $( gcloud projects describe $PROJECT_ID \
    --format = "value(projectNumber)" )
export SERVICE = "my-api"
export REGION_A = "africa-south1"
export REGION_B = "us-central1"
export DOMAIN = "api.yourdomain.com"
export IMAGE = "gcr.io/ ${ PROJECT_ID } / ${ SERVICE } :latest"

gcloud config set project $PROJECT_ID

# Enable required APIs
gcloud services enable \
    run.googleapis.com \
    compute.googleapis.com \
    artifactregistry.googleapis.com \
    cloudbuild.googleapis.com \
    networkservices.googleapis.com

# Grant Cloud Build service account the Cloud Run builder role
gcloud projects add-iam-policy-binding $PROJECT_ID \
    --member = "serviceAccount: ${ PROJECT_NUMBER } - c******@developer.gserviceaccount.com" \
    --role = "roles/run.builder"

Step 1: Implement the readiness probe endpoint

The first step and the most important one for Service Health to work, is adding a readiness probe endpoint to your application. Unlike the previous/alternative approach where the /health endpoint was for the load balancer's benefit, this endpoint is called directly by Cloud Run on each container instance to determine per-instance readiness.

Two rules from the official docs that matter here:

  • Use an HTTP/1 endpoint (the Cloud Run default, not HTTP/2)
  • The endpoint path must match the path in your probe configuration exactly
// Node.js / Express
// Lightweight - no DB calls, no downstream dependencies
// This runs frequently on every instance
app . get ( ' /health ' , ( req , res ) => {
    res . status ( 200 ). json ({ status : ' healthy ' , region : process . env . REGION , timestamp : new Date (). toISOString () });
});

// If you want the probe to reflect actual readiness
// (e.g. connection pool initialised), you can check internal state:
let isReady = false ;
app . get ( ' /health ' , ( req , res ) => {
    if ( ! isReady ) {
        return res . status ( 503 ). json ({ status : ' not_ready ' });
    }
    res . status ( 200 ). json ({ status : ' healthy ' , region : process . env . REGION });
});
// Set isReady = true after your startup tasks complete
pool . connect (). then (() => { isReady = true ; });

# Python / FastAPI
import os
from datetime import datetime
from fastapi import FastAPI , Response

app = FastAPI ()
is_ready = False

@app.get ( " /health " )
async def readiness_probe ( response : Response ):
    if not is_ready :
        response . status_code = 503
        return { " status " : " not_ready " }
    return { " status " : " healthy " , " region " : os . environ . get ( " REGION " ), " timestamp " : datetime . utcnow (). isoformat () }

@app.on_event ( " startup " )
async def startup_event ():
    global is_ready
    # Initialise connections, warm caches, etc.
    await init_database_pool ()
    is_ready = True

The is_ready pattern is the key upgrade over a basic /health endpoint. The readiness probe on each instance will return 503 until your startup tasks complete, preventing the load balancer from routing traffic to an instance that is running but not yet ready.

Step 2: Deploy to multiple regions with readiness probes

The gcloud run deploy supports deploying to multiple regions in a single command, and the --readiness-probe flag attaches the probe configuration at deploy time. Failovers require at least two (2) services from different regions.

# Deploy to both regions simultaneously with readiness probe
gcloud run deploy $SERVICE \
    --image = $IMAGE \
    --regions = $REGION_A , $REGION_B \
    --min = 1 \
    --max-instances = 100 \
    --concurrency = 80 \
    --cpu = 1 \
    --memory = 512Mi \
    --timeout = 30s \
    --readiness-probe = "httpGet.path=/health" \
    --set-env-vars = "ENV=production" \
    --allow-unauthenticated

The --readiness-probe="httpGet.path=/health" flag is the new way to configure probes at deploy time. You can also configure additional probe parameters:

# Full readiness probe configuration
gcloud run deploy $SERVICE \
    --image = $IMAGE \
    --regions = $REGION_A , $REGION_B \
    --min = 2 \
    --readiness-probe = "httpGet.path=/health,periodSeconds=10,failureThreshold=3,successThreshold=1,timeoutSeconds=5"

Or via YAML service definition (the Terraform-friendly approach):

# service.yaml
apiVersion : serving.knative.dev/v1
kind : Service
metadata :
  name : my-api
spec :
  template :
    metadata :
      annotations :
        autoscaling.knative.dev/minScale : " 1"
        autoscaling.knative.dev/maxScale : " 100"
    spec :
      containers :
      - image : gcr.io/PROJECT_ID/my-api:latest
        resources :
          limits :
            cpu : " 1"
            memory : 512Mi
        env :
        - name : ENV
          value : production
        readinessProbe :
          httpGet :
            path : /health
          periodSeconds : 10
          failureThreshold : 3
          successThreshold : 1
          timeoutSeconds : 5
        livenessProbe :
          httpGet :
            path : /health
          periodSeconds : 30
          failureThreshold : 3

The difference between readiness and liveness probes

Both probe types are supported on Cloud Run. Understanding the distinction is critical:

  • Readiness probe failure: Cloud Run stops routing requests to that instance. The instance continues running. Once the probe succeeds again, routing resumes. Service Health aggregates these to determine regional health.
  • Liveness probe failure: Cloud Run restarts the container instance. Use liveness probes for detecting deadlocks or unrecoverable stuck states.

For Service Health's automatic failover, readiness probes are what matter. Liveness probes are a complement - they handle instance-level recovery, while readiness probes handle traffic routing decisions.

Step 3: Set up the global external Application Load Balancer

With the new Service Health model, the load balancer configuration is simpler than before - you no longer need to configure a separate HTTPS health check at the load balancer level. Service Health exposes regional health through the Serverless NEG itself.

Create the backend service

# Single backend service, both regions are added as NEG backends
gcloud compute backend-services create $SERVICE -bs \
    --load-balancing-scheme = EXTERNAL_MANAGED \
    --global

Note: unlike the earlier approach with separate backend services per region, Service Health works with a single backend service that has multiple regional NEG backends. The load balancer reads health from each NEG and routes accordingly.

Reserve a global static IP

Set up a global static external IP address to reach your load balancer:

gcloud compute addresses create $SERVICE -ip \
    --network-tier = PREMIUM \
    --ip-version = IPV4 \
    --global

export GLOBAL_IP = $( gcloud compute addresses describe $SERVICE -ip \
    --global --format = "get(address)" )
echo "Global IP: ${ GLOBAL_IP } "
# → Update your DNS A record to this IP before proceeding

Create URL map, proxy, and forwarding rules

# Create a URL map to route incoming requests to the backend service:
gcloud compute url-maps create $SERVICE -lb \
    --default-service = $SERVICE -bs

# For HTTPS (recommended for production):
# Create Google-managed SSL certificate
gcloud compute ssl-certificates create $SERVICE -ssl \
    --domains = $DOMAIN \
    --global

# Create the target HTTPS proxy to route requests to your URL map:
gcloud compute target-https-proxies create $SERVICE -https-proxy \
    --url-map = $SERVICE -lb \
    --ssl-certificates = $SERVICE -ssl

# Create the HTTPS forwarding rule to route incoming requests to the proxy:
gcloud compute forwarding-rules create $SERVICE -https-fr \
    --load-balancing-scheme = EXTERNAL_MANAGED \
    --network-tier = PREMIUM \
    --address = $SERVICE -ip \
    --target-https-proxy = $SERVICE -https-proxy \
    --global \
    --ports = 443

# HTTP forwarding rule (redirect to HTTPS)
gcloud compute target-http-proxies create $SERVICE -http-proxy \
    --url-map = $SERVICE -lb

gcloud compute forwarding-rules create $SERVICE -http-fr \
    --load-balancing-scheme = EXTERNAL_MANAGED \
    --network-tier = PREMIUM \
    --address = $SERVICE -ip \
    --target-http-proxy = $SERVICE -http-proxy \
    --global \
    --ports = 80

Step 4: Create Serverless NEGs and attach them

# Serverless NEG for africa-south1
gcloud compute network-endpoint-groups create $SERVICE -neg- $REGION_A \
    --region = $REGION_A \
    --network-endpoint-type = serverless \
    --cloud-run-service = $SERVICE

# Serverless NEG for us-central1
gcloud compute network-endpoint-groups create $SERVICE -neg- $REGION_B \
    --region = $REGION_B \
    --network-endpoint-type = serverless \
    --cloud-run-service = $SERVICE

# Add both NEGs to the single backend service
gcloud compute backend-services add-backend $SERVICE -bs \
    --global \
    --network-endpoint-group = $SERVICE -neg- $REGION_A \
    --network-endpoint-group-region = $REGION_A

gcloud compute backend-services add-backend $SERVICE -bs \
    --global \
    --network-endpoint-group = $SERVICE -neg- $REGION_B \
    --network-endpoint-group-region = $REGION_B

At this point, Service Health is active. Cloud Run is running readiness probes on every instance in both regions, aggregating the results into a regional health signal, and the load balancer reads that signal via the Serverless NEGs.

Step 5: Monitor Service Health with Cloud Monitoring

Service Health exposes two metrics through Cloud Monitoring that you should track from day one:

  • run.googleapis.com/container/instance_count_with_readiness - the number of instances passing their readiness probe per region. Watch this metric to see the health state of your instance pool in each region in real time.
  • run.googleapis.com/service_health_count - the regional Cloud Run service health status.
Read on DEV Community ↗ ← Back to News

Comments

No comments yet. Start the discussion.