Assumes you're comfortable with Part 4's shared-responsibility framing (Compute Engine sitting at the "you manage everything" end) — this chapter moves progressively toward the "Google manages more" end of that same spectrum.
Table of Contents#
- What This Chapter Covers
- GKE Autopilot vs. Standard — and the 2026 Hybrid Option
- Deploying a GKE Cluster
- kubectl and Cluster Access
- Deploying a Containerized Application to GKE
- Node Pools — Standard Mode's Own Fleet Management
- Pod Autoscaling: HPA, VPA, and Autopilot's Resource Model
- GKE and Artifact Registry
- Cloud Run — Deploying and Managing Revisions
- Traffic Splitting, Gradual Rollouts, and Rollbacks
- Cloud Run Functions and Eventarc
- Choosing Between GKE, Cloud Run, and Compute Engine
- GPUs on Cloud Run and GKE
- A Full Worked Example: Meridian's Shipment API on Cloud Run
- A Parallel Full Worked Example: Meridian's route-optimizer on GKE
- Real-World Scenario: The Cold-Start Investigation
- Namespaces and Multi-Tenancy on GKE
- A Brief Word on Service Mesh
- GKE and Serverless Terminology Map
- Second Real-World Scenario: The Cluster Upgrade That Broke Nothing (On Purpose)
- Worked Cost Comparison: Autopilot vs. Standard vs. Cloud Run
- Pre-Flight Checklist: Is This Serverless/GKE Design Production-Ready?
- Chapter Recap: How the Pieces Connect
- Common Mistakes and Interview Traps
- Worked Practice Problems
- Summary and What's Next
What This Chapter Covers#
This chapter covers the two platforms GCP offers between raw Compute Engine and "not compute at all" — GKE (managed Kubernetes) and Cloud Run/Cloud Run functions (fully serverless) — and, more importantly, the judgment for choosing between them and Compute Engine for a given workload, which the ACE exam tests directly and production engineering tests even more.
🎯 By the end of this chapter, you'll be able to deploy a real application to both GKE and Cloud Run, configure autoscaling correctly on each, and make (and defend) the platform choice for a new workload based on its actual operational profile rather than familiarity or habit.
Where Part 4 put Meridian's GPS-ingestion fleet on raw Compute Engine deliberately — a workload whose interruption tolerance and steady load profile made that the right choice — this chapter's running examples are the two workloads Part 1's architecture diagram put elsewhere: shipment-api (the customer-facing tracking API, on Cloud Run) and a newer internal service, route-optimizer, which Ana's team is building on GKE specifically because it needs closer control over its runtime than Cloud Run's model allows.
GKE Autopilot vs. Standard — and the 2026 Hybrid Option#
GKE offers two cluster modes with a fundamentally different division of labor, and — as of 2026 — a third option that combines both inside a single cluster rather than forcing an all-or-nothing choice.
The 2026 hybrid option exists specifically for the common real case — most workloads in a cluster want Autopilot's simplicity, a few need Standard's control — that used to force picking one extreme for the whole cluster.
| Mode | Who manages nodes | Billing model | Reach for it when... |
|---|---|---|---|
| Autopilot | Google — provisioning, upgrades, OS, entirely hands-off | Per-pod resource consumption | Operational simplicity matters more than fine-grained node control; most standard-pattern workloads |
| Standard | You — full control over node pools, machine types, node-level configuration | Full VM capacity, whether used or idle | A workload needs privileged containers, custom node OS images, or elevated node access — Autopilot-only-mode's real, hard limitations |
| Standard + Autopilot ComputeClass (hybrid) | Mixed — you choose per workload | Mixed, matching each workload's own mode | A cluster with a mix of standard-pattern and specialized workloads — no longer an all-or-nothing decision |
# Create an Autopilot cluster — Meridian's default choice for any new
# GKE workload, absent a specific reason for Standard's extra control
gcloud container clusters create-auto route-optimizer-cluster \
--region=us-central1 \
--project=meridian-shipment-prod
# The 2026 hybrid: a Standard cluster where SPECIFIC workloads opt
# into Autopilot-managed nodes via a ComputeClass, while the rest of
# the cluster keeps hand-managed node pools
gcloud container clusters create route-optimizer-cluster-hybrid \
--region=us-central1 --enable-autoprovisioning# A Pod requesting the Autopilot ComputeClass inside an otherwise
# Standard cluster — GKE provisions and manages nodes for THIS
# workload specifically, while sibling workloads keep using the
# cluster's own hand-managed node pools
apiVersion: apps/v1
kind: Deployment
metadata:
name: route-optimizer
spec:
template:
spec:
nodeSelector:
cloud.google.com/compute-class: Autopilot
containers:
- name: route-optimizer
image: us-central1-docker.pkg.dev/meridian-shipment-prod/apps/route-optimizer:v3Tip
Best practice: default to Autopilot for new clusters, and reach for Standard (or the hybrid ComputeClass approach) only once a specific, named requirement rules Autopilot out — privileged containers, DaemonSets needing elevated node access, or a genuinely cost-driven need for sustained high utilization above roughly 60-70%, where Standard's "pay for the full VM" model becomes cheaper than Autopilot's per-pod billing. Choosing Standard by default "for control we might need later" repeats the same over-provisioning-by-caution mistake Part 4 called out for machine-family selection.
From the Trenches: The Autopilot Limitation Discovered Mid-Migration#
A team migrating a legacy monitoring agent's DaemonSet onto GKE Autopilot discovered, partway through the migration, that Autopilot doesn't support DaemonSets requiring privileged, node-level access the way their existing agent needed — a hard platform limitation, not a misconfiguration to work around. The immediate fix required falling back to Standard mode for that one workload; the deeper lesson was that the decision between Autopilot and Standard should be made by checking a workload's actual requirements against Autopilot's documented restrictions before migration starts, not discovered mid-migration — exactly the kind of check the 2026 hybrid ComputeClass option now makes far less costly to get wrong, since a single workload needing Standard-only capabilities no longer forces the entire cluster onto Standard mode.
Deploying a GKE Cluster#
# A Standard-mode cluster with explicit configuration, for
# comparison against the Autopilot command above — this level of
# detail (node count, machine type, network) simply doesn't exist
# as a decision on Autopilot, since Google manages all of it
gcloud container clusters create route-optimizer-standard \
--zone=us-central1-a \
--num-nodes=3 \
--machine-type=n4-standard-4 \
--enable-ip-alias \
--workload-pool=meridian-shipment-prod.svc.id.googThe --workload-pool flag is worth flagging immediately, since it's the setup step Part 3's Workload Identity Federation for GKE section depends on — without it, Pods in this cluster have no way to authenticate as a GCP service account at all, forcing a fallback to the node's own default service account (repeating Part 4's "default identity, shared blast radius" warning, one layer up at the cluster level).
Private clusters — where nodes have no public IP at all, matching Part 1's org-policy default — are the production standard, not an optional hardening step:
gcloud container clusters create-auto route-optimizer-cluster \
--region=us-central1 \
--enable-private-nodes \
--master-ipv4-cidr=172.16.0.0/28kubectl and Cluster Access#
# Install/configure kubectl for GKE specifically — the gke-gcloud-auth-plugin
# bridges gcloud's own IAM-based auth into kubectl's credential model
gcloud components install gke-gcloud-auth-plugin
# Fetch cluster credentials — this is what actually lets kubectl
# talk to a specific cluster using your current gcloud identity
gcloud container clusters get-credentials route-optimizer-cluster \
--region=us-central1
# Confirm access
kubectl get nodes
kubectl config current-contextAccess to a GKE cluster is governed by the same IAM system Part 3 built in full — roles/container.developer for someone who deploys workloads, roles/container.admin for full cluster management — layered underneath Kubernetes' own RBAC, which can further restrict what an authenticated identity can do within the cluster even after IAM has let them connect at all. Meridian keeps this simple by mapping IAM roles to broad access tiers and reserving Kubernetes RBAC for finer per-namespace restrictions once route-optimizer grows beyond a single team's ownership.
Deploying a Containerized Application to GKE#
# route-optimizer-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: route-optimizer
labels:
team: data
env: prod
spec:
replicas: 3
selector:
matchLabels:
app: route-optimizer
template:
metadata:
labels:
app: route-optimizer
spec:
serviceAccountName: route-optimizer-ksa
containers:
- name: route-optimizer
image: us-central1-docker.pkg.dev/meridian-shipment-prod/apps/route-optimizer:v3
resources:
requests:
cpu: "500m"
memory: "1Gi"
limits:
cpu: "1000m"
memory: "2Gi"
ports:
- containerPort: 8080kubectl apply -f route-optimizer-deployment.yaml
kubectl rollout status deployment/route-optimizerReadiness and Liveness Probes — Kubernetes' Own Health Checks#
Kubernetes has its own health-check mechanism, conceptually parallel to Part 4's Compute Engine health checks but operating at the Pod level rather than the VM level — and the same "an under-specified check causes more harm than no check" lesson from Part 4 applies here just as directly.
# Added to the route-optimizer-deployment.yaml container spec
livenessProbe:
httpGet:
path: /healthz
port: 8080
initialDelaySeconds: 30
periodSeconds: 15
readinessProbe:
httpGet:
path: /ready
port: 8080
initialDelaySeconds: 10
periodSeconds: 10| Probe | Answers | Consequence of failure |
|---|---|---|
| Liveness | "Is this Pod stuck/deadlocked and needs a restart?" | Kubernetes restarts the container |
| Readiness | "Is this Pod ready to receive traffic right now?" | The Pod is removed from Service load-balancing until it passes again — no restart |
Conflating the two is a genuinely common mistake: a Pod that's temporarily busy (say, running a slow startup dependency check) should fail readiness — briefly removed from traffic, no restart needed — not liveness, which would restart a Pod that was never actually broken, just temporarily unready. Meridian's route-optimizer deliberately uses separate endpoints (/healthz for liveness, /ready for readiness) specifically because they check genuinely different things: /healthz confirms the process itself hasn't deadlocked, while /ready additionally confirms its database connection pool has finished initializing — a Pod can be alive (not deadlocked) while still legitimately not ready for traffic during its first few seconds.
Warning
Setting initialDelaySeconds too low on a liveness probe for a workload with real startup time repeats Part 4's own patch-job cascade lesson one layer up: Kubernetes restarts a Pod that was simply still starting, which can produce a genuine restart loop — the Pod never finishes starting because it keeps getting killed mid-startup by an impatient liveness check.
Important
On GKE Autopilot specifically, resource requests aren't just a scheduling hint — they directly determine billing, since Autopilot bills per-pod based on requested (not necessarily used) CPU/memory. Setting requests far higher than the workload actually needs "to be safe" has a direct, immediate cost consequence on Autopilot in a way it doesn't on Standard mode, where the cost is already sunk into whatever VMs the node pool runs regardless of individual pod requests.
Node Pools — Standard Mode's Own Fleet Management#
A node pool is Standard mode's equivalent of Part 4's managed instance group — a set of identically-configured nodes, with its own machine type, autoscaling range, and upgrade policy, independent of other node pools in the same cluster.
# Add a second node pool with different sizing — e.g., a node pool
# for memory-heavy workloads, alongside a cluster's original
# general-purpose pool
gcloud container node-pools create memory-optimized-pool \
--cluster=route-optimizer-standard \
--zone=us-central1-a \
--machine-type=n4-highmem-4 \
--num-nodes=2 \
--enable-autoscaling --min-nodes=1 --max-nodes=6# Editing or removing a node pool — every node in it, add/edit/remove
gcloud container node-pools list --cluster=route-optimizer-standard --zone=us-central1-a
gcloud container node-pools delete legacy-pool --cluster=route-optimizer-standard --zone=us-central1-aMultiple node pools let a Standard cluster host workloads with genuinely different resource profiles without either over-provisioning a single pool's machine type for the least-demanding workload, or under-provisioning it for the most-demanding one — the same "match the machine type to the measured profile" discipline from Part 4, applied per workload category rather than per whole cluster.
Pod Autoscaling: HPA, VPA, and Autopilot's Resource Model#
Three distinct autoscaling mechanisms exist inside Kubernetes, each scaling a different dimension, and confusing them is a common source of "why isn't this scaling" confusion.
| Mechanism | Scales | Available on |
|---|---|---|
| Horizontal Pod Autoscaler (HPA) | The number of Pod replicas | Both Autopilot and Standard |
| Vertical Pod Autoscaler (VPA) | A Pod's own CPU/memory requests | Both, though interacts differently with Autopilot's billing model |
| Cluster Autoscaler (node-level) | The number of nodes in a Standard node pool | Standard only — Autopilot handles this invisibly as part of its own management |
# horizontal-pod-autoscaler.yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: route-optimizer-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: route-optimizer
minReplicas: 3
maxReplicas: 20
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 65The core conceptual mapping worth holding onto from Part 4: HPA is directly analogous to a managed instance group's autoscaling policy (more/fewer copies of the same unit), while the Cluster Autoscaler on Standard mode is analogous to Part 4's own instance-group-level scaling one layer down (more/fewer nodes to host those Pods) — Autopilot collapses that second layer away entirely, which is exactly the operational-simplicity trade-off the earlier decision table described.
GKE and Artifact Registry#
# Create a repository for container images — the modern replacement
# for the older Container Registry, which is being phased toward
# Artifact Registry across GCP's own tooling and defaults
gcloud artifacts repositories create apps \
--repository-format=docker \
--location=us-central1
# Build and push — the image reference used in the Deployment YAML above
gcloud builds submit --tag us-central1-docker.pkg.dev/meridian-shipment-prod/apps/route-optimizer:v3
# GKE pulls from Artifact Registry using the node's own service
# account identity — grant roles/artifactregistry.reader to
# whichever service account nodes run as, not to individual Pods
gcloud artifacts repositories add-iam-policy-binding apps \
--location=us-central1 \
--member="serviceAccount:route-optimizer-node-sa@meridian-shipment-prod.iam.gserviceaccount.com" \
--role="roles/artifactregistry.reader"Cloud Run — Deploying and Managing Revisions#
Cloud Run runs a stateless container behind an HTTPS endpoint, scaling automatically (including to zero) with no cluster, no node pool, and no Kubernetes API to manage at all — the far end of Part 1's shared-responsibility spectrum from Compute Engine.
# Deploy a new revision — every deploy creates a NEW, immutable
# revision rather than mutating a running one, the mechanism the
# next section's traffic splitting depends on
gcloud run deploy shipment-api \
--image=us-central1-docker.pkg.dev/meridian-shipment-prod/apps/shipment-api:v12 \
--region=us-central1 \
--service-account=shipment-api@meridian-shipment-prod.iam.gserviceaccount.com \
--min-instances=1 \
--max-instances=50 \
--no-allow-unauthenticated--min-instances=1 is a deliberate choice, not a default — Cloud Run's ability to scale to zero saves cost for genuinely idle services, but scaling from zero introduces a cold start (covered in this chapter's closing scenario) that a customer-facing API like shipment-api can't tolerate on its first request after an idle period. Meridian keeps one warm instance always running specifically to avoid that cold start on the customer-facing path, while an internal, rarely-used admin tool elsewhere in the org runs with --min-instances=0 because occasional cold starts there are a genuinely acceptable trade-off for not paying for constant idle capacity.
Traffic Splitting, Gradual Rollouts, and Rollbacks#
Every Cloud Run deployment creates a new revision without automatically routing any traffic to it — traffic assignment is a separate, explicit step, which is what makes gradual rollouts and instant rollbacks both possible.
# Deploy WITHOUT shifting traffic yet — the new revision exists but
# serves nothing until explicitly assigned traffic
gcloud run deploy shipment-api \
--image=us-central1-docker.pkg.dev/meridian-shipment-prod/apps/shipment-api:v13 \
--region=us-central1 --no-traffic --tag=canary
# Send 10% of traffic to the new revision, keep 90% on the
# previous one — a real canary, not an all-or-nothing switch
gcloud run services update-traffic shipment-api \
--region=us-central1 \
--to-revisions=shipment-api-v13=10,shipment-api-v12=90
# Once confidence is established, shift fully
gcloud run services update-traffic shipment-api \
--region=us-central1 --to-latest
# Instant rollback — traffic reassignment is the mechanism, no
# redeploy of the old code required since the old revision still exists
gcloud run services update-traffic shipment-api \
--region=us-central1 --to-revisions=shipment-api-v12=100Both revisions exist simultaneously and keep running until traffic is explicitly reassigned — a rollback is a traffic-management change, never a redeploy.
Tip
Best practice: default to a canary percentage for any change to a customer-facing Cloud Run service, and use --no-traffic plus a tagged URL to test a new revision directly before it receives any production traffic at all. The tagged URL (https://canary---shipment-api-xyz.a.run.app) lets Devon's team run real smoke tests against the exact new revision before a single percent of real customer traffic ever reaches it.
Concurrency and CPU Allocation — Sizing a Cloud Run Instance Correctly#
Two settings determine how many simultaneous requests one Cloud Run instance handles, and getting them wrong either wastes money or silently degrades latency under load.
# --concurrency: how many requests one instance handles AT ONCE
# --cpu: how many vCPUs allocated per instance
# --cpu-throttling / --no-cpu-throttling: whether CPU is available
# only during request handling, or continuously (needed for
# background work between requests)
gcloud run deploy shipment-api \
--image=us-central1-docker.pkg.dev/meridian-shipment-prod/apps/shipment-api:v13 \
--region=us-central1 \
--concurrency=80 \
--cpu=2 --memory=2Gi \
--no-cpu-throttling| Setting | Too low | Too high |
|---|---|---|
--concurrency | Cloud Run creates more instances than necessary for the same load — wastes the per-instance minimum cost | A single slow request can block others sharing that instance, degrading latency for requests that would otherwise be fast |
--cpu | Requests queue or slow down under real load, since the instance genuinely lacks compute | Paying for CPU capacity the workload never uses |
Meridian tuned shipment-api's concurrency down from Cloud Run's default of 80 to a measured 40 after finding that above that point, tail latency (the slowest 5% of requests, not the average) degraded noticeably during peak traffic — a database-connection-bound workload doesn't necessarily benefit from as much per-instance concurrency as a purely CPU-bound one would, since the bottleneck isn't the container's own CPU at all.
Tip
Best practice: don't accept Cloud Run's default concurrency value without testing against your actual workload's bottleneck. A CPU-bound workload might genuinely handle 80+ concurrent requests per instance well; a workload bottlenecked on a downstream dependency (a database connection pool, an external API's own rate limit) often needs a materially lower concurrency setting to keep tail latency acceptable, regardless of how much spare CPU the instance has.
Cloud Run Functions and Eventarc#
Cloud Run functions (the current generation, replacing the older "Cloud Functions" branding) share Cloud Run's own revision and traffic-splitting model, and connect to events through Eventarc — a unified routing layer covering more than 90 event sources, not just a Cloud Functions-specific trigger mechanism.
# Deploy a Cloud Run function triggered by a Cloud Storage event via Eventarc
gcloud functions deploy process-shipment-document \
--gen2 \
--runtime=python312 \
--region=us-central1 \
--trigger-event-filters="type=google.cloud.storage.object.v1.finalized" \
--trigger-event-filters="bucket=meridian-shipment-documents" \
--service-account=doc-processor@meridian-shipment-prod.iam.gserviceaccount.comBecause Cloud Run functions share Cloud Run's underlying revision model, the same traffic-splitting and instant-rollback mechanism from the previous section applies identically here — a genuinely useful consistency, since a team fluent in one doesn't have to relearn deployment safety mechanics for the other.
Choosing Between GKE, Cloud Run, and Compute Engine#
Bringing this chapter's platform choice together with Part 4's Compute Engine coverage, into the single decision framework the ACE exam and real architecture reviews both actually need:
| Workload shape | Best fit | Why |
|---|---|---|
| Stateless HTTP service, spiky or unpredictable traffic, no special runtime needs | Cloud Run | Scales to zero, no cluster to manage, fastest path from container to production |
| Event-driven processing triggered by a GCP event source | Cloud Run functions + Eventarc | Purpose-built for exactly this trigger-and-respond shape |
| A workload needing fine-grained control over networking, sidecars, or complex multi-container orchestration | GKE (Autopilot first, Standard if a specific limitation requires it) | Kubernetes' own primitives (sidecars, DaemonSets, custom schedulers) aren't available on Cloud Run at all |
| A workload needing privileged access, custom kernel modules, or GPU/TPU configurations Cloud Run doesn't yet support | GKE Standard, or Compute Engine directly | The two platforms retaining full node-level control |
| A steady-state, long-running batch or ingestion workload with predictable resource needs | Compute Engine (Part 4) | The most cost-efficient at sustained, predictable utilization, especially combined with Spot capacity |
A quick reference for which section of this chapter to revisit for a specific real-world symptom, the same style of table Part 4 closed with:
| Symptom | Revisit |
|---|---|
| Fleet of Pods won't schedule, cluster "seems full" | Namespaces and Multi-Tenancy — check ResourceQuota, not just raw cluster capacity |
| Occasional slow first request after quiet periods | Cloud Run min-instances and cold starts |
| A Pod keeps restarting during what should be normal startup | Readiness/Liveness probe timing |
| Tail latency degrades under load despite spare CPU | Cloud Run concurrency setting |
| A cluster upgrade feels risky every time | Node pool surge/unavailable upgrade parameters |
| Unsure which platform a new workload belongs on | The GKE/Cloud Run/Compute Engine decision table |
Meridian's own three-way split validates this table directly: shipment-api (spiky, customer-facing HTTP) on Cloud Run, route-optimizer (needing closer runtime control as its ML dependencies grow) on GKE, and the GPS-ingestion fleet (steady, interruption-tolerant) on Compute Engine — three different platforms for three genuinely different workload shapes, not an arbitrary or inconsistent choice.
GPUs on Cloud Run and GKE#
GPU support has extended into Cloud Run as of 2026, including newer NVIDIA hardware generations — meaning a GPU-dependent inference workload no longer automatically requires GKE or Compute Engine the way it did in earlier years.
# A Cloud Run service with an attached GPU — genuinely useful for
# a lightweight inference workload that still wants Cloud Run's
# scale-to-zero and no-cluster-to-manage model
gcloud run deploy route-optimizer-inference \
--image=us-central1-docker.pkg.dev/meridian-shipment-prod/apps/inference:v1 \
--region=us-central1 \
--gpu=1 --gpu-type=nvidia-l4Note
GPU-attached Cloud Run instances don't scale to zero as cheaply or as instantly as a CPU-only service — a real cold-start and cost consideration specific to this configuration, worth confirming against current documentation before assuming Cloud Run's usual "pay only when handling requests" framing applies identically once a GPU is attached.
A Full Worked Example: Meridian's Shipment API on Cloud Run#
# 1. A scoped service account (Part 3), already created
# 2. Deploy with production-appropriate settings
gcloud run deploy shipment-api \
--image=us-central1-docker.pkg.dev/meridian-shipment-prod/apps/shipment-api:v13 \
--region=us-central1 \
--service-account=shipment-api@meridian-shipment-prod.iam.gserviceaccount.com \
--min-instances=1 --max-instances=50 \
--no-allow-unauthenticated \
--vpc-connector=meridian-serverless-connector \
--set-env-vars="ENV=production"
# 3. A canary rollout, per this chapter's traffic-splitting section
gcloud run services update-traffic shipment-api \
--region=us-central1 \
--to-revisions=shipment-api-v13=10,shipment-api-v12=90
# 4. Confirm the canary's own error rate and latency before promoting —
# Part 8 covers exactly how to read this from Cloud Monitoring
gcloud monitoring time-series list \
--filter='resource.type="cloud_run_revision" AND resource.label.revision_name="shipment-api-v13"'
# 5. Promote to 100% once confirmed healthy
gcloud run services update-traffic shipment-api --region=us-central1 --to-latestThe --vpc-connector flag matters specifically because shipment-api needs to reach Cloud SQL over a private IP (Part 6 covers this connection in depth) — Cloud Run's serverless networking is otherwise isolated from a VPC by default, another concrete instance of the shared-responsibility trade-off this chapter keeps returning to: Cloud Run hides almost all infrastructure concerns, and a VPC connector is the deliberate, explicit bridge back to the private networking Part 1 and Part 7 build everything else around.
A Parallel Full Worked Example: Meridian's route-optimizer on GKE#
Mirroring the Cloud Run worked example above, here's the equivalent end-to-end setup for route-optimizer on GKE — worth reading side by side with the Cloud Run version, since the shape of the work (identity, deployment, gradual rollout, verification) is similar even though the mechanics differ substantially:
# 1. A private Autopilot cluster with Workload Identity Federation
# configured (Part 3) from the start, not retrofitted later
gcloud container clusters create-auto route-optimizer-cluster \
--region=us-central1 \
--enable-private-nodes \
--workload-pool=meridian-shipment-prod.svc.id.goog
# 2. Bind the Kubernetes ServiceAccount to the GCP service account —
# the exact binding introduced in Part 3's GKE Workload Identity section
gcloud iam service-accounts add-iam-policy-binding \
route-optimizer@meridian-shipment-prod.iam.gserviceaccount.com \
--role="roles/iam.workloadIdentityUser" \
--member="serviceAccount:meridian-shipment-prod.svc.id.goog[route-optimizer/route-optimizer-ksa]"
# 3. Apply the namespace, resource quota, and Deployment together
kubectl apply -f route-optimizer-namespace.yaml
kubectl apply -f route-optimizer-quota.yaml
kubectl apply -f route-optimizer-deployment.yaml
# 4. Apply the HorizontalPodAutoscaler from earlier in this chapter
kubectl apply -f horizontal-pod-autoscaler.yaml
# 5. A gradual rollout via Kubernetes' own native mechanism — a
# Deployment's rollout is progressive by default (maxSurge/maxUnavailable),
# the GKE-native analog of Cloud Run's explicit traffic percentages
kubectl set image deployment/route-optimizer \
route-optimizer=us-central1-docker.pkg.dev/meridian-shipment-prod/apps/route-optimizer:v4 \
--namespace=route-optimizer
kubectl rollout status deployment/route-optimizer --namespace=route-optimizer
# 6. Rollback, if needed — Kubernetes keeps rollout history natively,
# the GKE-native equivalent of Cloud Run's revision-based rollback
kubectl rollout undo deployment/route-optimizer --namespace=route-optimizerNote
Step 6's rollout undo and Cloud Run's update-traffic --to-revisions solve the identical underlying problem (get back to known-good code fast) through genuinely different mechanisms — Kubernetes rolls Pods back to a previous ReplicaSet's spec, which takes a real (if usually brief) amount of time to reschedule and become ready, while Cloud Run's rollback is a pure routing change against a revision that's already running. Neither is "the correct" rollback mechanism in the abstract — it's simply what each platform's own architecture provides, and worth knowing the distinction when moving between the two.
Real-World Scenario: The Cold-Start Investigation#
Before Meridian set --min-instances=1 on shipment-api, Ana's team noticed a recurring pattern in customer support tickets: occasional, unexplained multi-second delays on the very first tracking request after a period of low overnight traffic, with every subsequent request fast. The investigation traced it to Cloud Run scaling shipment-api down to zero instances during the lowest-traffic overnight window, then needing to cold-start a new instance (container pull, application boot, dependency initialization) the moment the first morning request arrived — a real, measurable delay invisible in any metric that only looks at average latency, since cold starts affected a tiny fraction of requests but produced a genuinely bad experience for whichever customer happened to trigger one.
Average latency metrics smooth right over this — the fix required looking at latency percentiles and correlating spikes with instance-count drops to zero, not just watching a mean.
The fix was --min-instances=1, accepting the small, predictable cost of one always-on instance in exchange for eliminating the unpredictable, customer-visible cold start entirely — the same kind of explicit, deliberate cost-for-reliability trade-off Part 4 made when choosing a standard-VM floor over an all-Spot fleet. Scale-to-zero's cost savings and cold-start risk are two sides of the same design decision, not a free win — the right choice depends entirely on whether the specific workload can tolerate an occasional multi-second delay on its first request after idle time, and a customer-facing API usually can't.
Namespaces and Multi-Tenancy on GKE#
A Kubernetes namespace partitions a single cluster into logically separate areas — its own name scoping, its own resource quotas, its own RBAC boundary — without the overhead of running a separate cluster per team or workload.
# Create a namespace per team, mirroring the project-per-environment
# discipline from Part 1 one level down, inside a single cluster
kubectl create namespace gps-ingestion
kubectl create namespace route-optimizer
# A resource quota scoping how much of the cluster's capacity one
# namespace can consume — prevents one team's workload from starving
# another's, the in-cluster equivalent of Part 1's per-project quotas
kubectl apply -f - <<EOF
apiVersion: v1
kind: ResourceQuota
metadata:
name: route-optimizer-quota
namespace: route-optimizer
spec:
hard:
requests.cpu: "8"
requests.memory: 16Gi
limits.cpu: "16"
limits.memory: 32Gi
EOF# RBAC scoped to exactly one namespace — a data-team member gets
# full access within route-optimizer's namespace, and NOTHING
# outside it, without needing a separate cluster to enforce that boundary
kubectl create rolebinding data-team-route-optimizer \
--clusterrole=edit \
--group=data-team@meridianlogistics.com \
--namespace=route-optimizerTip
Best practice: default to namespace-per-team or namespace-per-workload inside a shared cluster, reserving a genuinely separate cluster for cases needing a harder isolation boundary (a different compliance scope, a genuinely different network policy, or a team that shouldn't even be able to see another team's workloads exist). Namespaces solve resource and RBAC isolation cheaply; a separate cluster solves it more completely but at real operational cost — running, upgrading, and monitoring N clusters instead of one.
From the Trenches: The Missing Resource Quota#
Before Meridian added the ResourceQuota shown above, a runaway batch job Ana's team deployed into the route-optimizer namespace during a debugging session consumed enough of the shared cluster's capacity that gps-ingestion's own Pods (in a separate namespace, but the same physical cluster) started failing to schedule — a resource-starvation incident with no namespace boundary actually stopping it, since namespaces alone don't cap consumption without an explicit quota attached. The immediate cause was one namespace's workload consuming shared capacity unbounded; the deeper lesson is that a namespace provides a naming and RBAC boundary by default, but not a resource boundary — that requires an explicit ResourceQuota, easy to assume is automatic and genuinely isn't.
A Brief Word on Service Mesh#
A service mesh (Istio, or GKE's own managed Cloud Service Mesh) adds a dedicated infrastructure layer for service-to-service traffic — mutual TLS between services, fine-grained traffic routing beyond what a basic Kubernetes Service provides, and rich per-request observability — sitting alongside, not replacing, everything this chapter covered. Meridian doesn't run a service mesh today; route-optimizer is currently the only substantial GKE workload, and a mesh's real value (traffic management and security between many services) doesn't pay for its operational complexity until a cluster hosts enough interacting services to need it. This site's Containers & Orchestration domain (specifically the OpenShift/Service Mesh coverage) goes deep on this topic — worth a look once a GKE footprint grows past a handful of services, not before.
GKE and Serverless Terminology Map#
| Concept | GCP | AWS | Azure |
|---|---|---|---|
| Managed Kubernetes | GKE (Autopilot / Standard) | EKS (with Fargate for a similar hands-off mode) | AKS |
| Fully serverless containers | Cloud Run | App Runner / Fargate | Azure Container Apps |
| Event-driven functions | Cloud Run functions | Lambda | Azure Functions |
| Unified event routing | Eventarc | EventBridge | Event Grid |
| Container image registry | Artifact Registry | Elastic Container Registry (ECR) | Azure Container Registry |
Where this mapping holds up well: the conceptual shapes (managed Kubernetes, a serverless container platform, an event router) are genuinely comparable across all three clouds — this is one of the more consistent cross-provider mappings in this course, since containers themselves are a portable, standardized technology in a way IAM and resource hierarchies never were.
Second Real-World Scenario: The Cluster Upgrade That Broke Nothing (On Purpose)#
When GKE announced a mandatory version upgrade deadline for the Kubernetes minor version route-optimizer's Standard cluster was running, Devon's team used the opportunity to validate the whole upgrade path end to end rather than accepting GKE's default auto-upgrade timing blind. The actual sequence, worth showing in full since "just let GKE auto-upgrade" and "carefully validate every upgrade" are both real, defensible positions depending on a workload's criticality:
# 1. Check the current version and available upgrade targets
gcloud container clusters describe route-optimizer-standard \
--zone=us-central1-a --format="value(currentMasterVersion)"
gcloud container get-server-config --zone=us-central1-a
# 2. Upgrade the CONTROL PLANE first, during a low-traffic window —
# GKE requires this before any node pool can upgrade to match
gcloud container clusters upgrade route-optimizer-standard \
--zone=us-central1-a --master --cluster-version=1.31
# 3. Upgrade node pools with a surge strategy — new nodes join
# BEFORE old ones are removed, so capacity never dips during the upgrade
gcloud container clusters upgrade route-optimizer-standard \
--zone=us-central1-a --node-pool=default-pool \
--max-surge-upgrade=1 --max-unavailable-upgrade=0
# 4. Confirm workload health throughout — this is what actually
# validates the upgrade, not just watching the version number change
kubectl get pods -n route-optimizer -wThe --max-surge-upgrade=1 --max-unavailable-upgrade=0 combination is the specific decision that mattered: it guarantees at least the original node count stays available throughout the entire upgrade, at the cost of briefly running one extra node during the transition — a small, bounded cost in exchange for the upgrade being genuinely invisible to route-optimizer's own availability. The team confirmed via the health-check-driven monitoring in step 4 that Pod scheduling and response latency stayed flat throughout the whole multi-hour rolling upgrade, turning a mandatory, deadline-driven upgrade into a validated non-event rather than a maintenance-window gamble.
Tip
Best practice: treat a GKE cluster's upgrade path the same way Part 4 treated a VM Manager patch job — test the specific rolling-update parameters against a non-production cluster first, and never let "GKE handles upgrades automatically" become a substitute for actually confirming your own workload survives one. GKE's automation handles the mechanics; validating your specific workload's behavior during the mechanics is still your job.
Worked Cost Comparison: Autopilot vs. Standard vs. Cloud Run#
Putting real, illustrative numbers behind the platform-choice decision table earlier in this chapter — always confirm current rates via the official pricing calculator before a real budget decision, but the relative shape below holds consistently:
| Scenario | Workload profile | Approx. monthly cost driver |
|---|---|---|
route-optimizer on GKE Autopilot, 3 replicas, 500m CPU / 1Gi memory each, running continuously | Steady, moderate, always-on | Per-pod resource billing — roughly $80-120/month per replica at this size, illustrative |
The same workload on GKE Standard, 3-node n4-standard-4 pool at ~40% utilization | Same workload, sized generously | Full VM cost regardless of the 60% idle capacity — often more than Autopilot below the 60-70% utilization crossover point cited earlier in this chapter |
shipment-api on Cloud Run, min-instances=1, spiky traffic averaging low CPU | Spiky, mostly idle outside business hours | One always-on instance's baseline cost, plus per-request billing only during actual traffic — typically cheapest for a genuinely spiky profile |
The crossover point cited earlier (Standard becomes cheaper above roughly 60-70% sustained utilization) is exactly why route-optimizer's current moderate, unpredictable load profile makes Autopilot the right cost choice today — and exactly the kind of number worth re-checking if the workload's profile changes materially, the same "revisit, don't set once" discipline this course has applied to IAM grants, org-policy exceptions, and golden images alike.
Pre-Flight Checklist: Is This Serverless/GKE Design Production-Ready?#
- GKE mode chosen (Autopilot by default) based on a specific, named requirement, not habit
-
--workload-poolconfigured on every GKE cluster — no Pod relying on the node's default identity - Private clusters/nodes used by default, matching Part 1's no-public-IP org policy
- Resource requests on Autopilot reflect real measured need, not an inflated "just in case" guess
- Every Cloud Run deployment goes through a canary/traffic-split step before full promotion
-
--min-instancesset deliberately per service based on cold-start tolerance, not left at the scale-to-zero default everywhere - Cloud Run functions triggered via Eventarc use a scoped service account, per Part 3
- The GKE vs. Cloud Run vs. Compute Engine choice for each new workload is justified against this chapter's decision table, not defaulted to whichever platform the team already knows best
Common Mistakes and Interview Traps#
| Mistake | Why it happens | The fix |
|---|---|---|
| Choosing GKE Standard by default "for control we might need later" | Control feels like the safer choice | Default to Autopilot; move to Standard (or the hybrid ComputeClass) only for a specific, named limitation |
| Setting Autopilot resource requests far above real need | Feels like a safety margin | Autopilot bills per-pod-request directly — an inflated request has an immediate, ongoing cost |
| Assuming Cloud Run's scale-to-zero has no user-facing cost | The cost savings are the visible part | Cold starts are a real, measurable latency cost for the first request after idle — set min-instances deliberately for latency-sensitive services |
| Shifting 100% of traffic to a new Cloud Run revision immediately | It's the simplest deploy command | Use a canary percentage and a tagged URL to test before full promotion — rollback is instant, but a canary catches problems before most users see them |
| Assuming a GPU-dependent workload automatically needs GKE or Compute Engine | Historically true, no longer the full picture | Cloud Run now supports GPU attachment for the right workload shape — check current platform capability before assuming |
| Confusing HPA, VPA, and the Cluster Autoscaler | All three are "Kubernetes autoscaling" | Each scales a different dimension (replicas, per-pod resources, node count) — know which one addresses a given symptom |
| Deploying multiple teams' workloads into a shared cluster with no ResourceQuota | Namespaces feel like a complete isolation boundary on their own | A namespace provides naming/RBAC isolation by default, not resource isolation — add an explicit ResourceQuota per namespace |
| Treating "GKE handles upgrades automatically" as sufficient validation | The automation genuinely handles the mechanics | Confirm your own workload's health throughout an upgrade — the mechanics working doesn't guarantee your application does |
| Adopting a service mesh before a cluster has enough interacting services to need one | Mesh features (mTLS, fine-grained routing) sound valuable in the abstract | The operational overhead isn't justified until service-to-service traffic complexity genuinely requires it — a single-service cluster gets little from a mesh |
Chapter Recap: How the Pieces Connect#
Worked Practice Problems#
1. Ana's team is building route-optimizer, a workload needing a DaemonSet with privileged node access for a specialized network monitoring tool, alongside several standard-pattern microservices in the same cluster. What's the 2026-era recommendation, and why would forcing the whole cluster onto Standard mode be the wrong call?
The 2026 hybrid option: keep the cluster on Standard mode overall (required for the DaemonSet's privileged access, which Autopilot doesn't support), but let the standard-pattern microservices opt into the Autopilot ComputeClass individually. Forcing the entire cluster onto Standard mode purely because one workload needs it would mean every other workload loses Autopilot's operational simplicity and per-pod billing model for no reason specific to those workloads — the hybrid option exists specifically so one workload's hard requirement doesn't dictate the operational model for workloads that don't share that requirement.
2. shipment-api runs with --min-instances=0 in a staging environment (to save cost) and --min-instances=1 in production. A QA engineer files a bug that staging "randomly" feels slow on the first request of the day but is otherwise fine. Is this a real bug, and what's the actual trade-off being made?
Not a bug — this is scale-to-zero's cold-start behavior working exactly as designed, and it's a deliberate, reasonable trade-off for a staging environment where occasional first-request latency is an acceptable cost for not paying for a constantly-running idle instance. The distinction from production (--min-instances=1) isn't an inconsistency to fix; it's the correct application of this chapter's own principle that the right min-instances setting depends on cold-start tolerance, and staging's tolerance for an occasional slow first request is genuinely higher than a customer-facing production API's.
3. A Cloud Run canary deployment sends 10% of traffic to a new revision. Five minutes in, the canary revision's error rate is elevated. What's the fastest correct response, and why is this specifically safer than a typical Compute Engine or GKE rollback?
Reassign 100% of traffic back to the previous stable revision (--to-revisions=shipment-api-v12=100) — an instant traffic-management change, not a redeploy, since the previous revision never stopped existing or running. This is specifically safer/faster than a typical VM-based or Kubernetes-Deployment-based rollback because those often require either recreating instances from an older template (Part 4) or rolling back a Deployment's Pod spec and waiting for new Pods to schedule and become ready — both real operations with their own latency, versus Cloud Run's rollback being purely a routing change against two revisions that are both already running.
4. route-optimizer's Autopilot cluster is at moderate, steady utilization today, but Ana projects it will grow to sustained high utilization (well above 70%) within two quarters as the ML model's usage scales. Should Meridian migrate to Standard mode now, later, or not at all — and what does this chapter's own guidance say about making that call?
Not now — Autopilot is the correct choice for the workload's current profile, and migrating preemptively based on a projection repeats the same over-provisioning-by-caution mistake this course has warned against in Part 4 (machine-family sizing) and earlier in this chapter (GKE mode selection). The right call is to revisit the decision once utilization actually crosses the cost-crossover point this chapter's worked cost comparison identified — treating the platform choice the same way this course treats IAM grants, org-policy exceptions, and golden images: correct for the situation at the time it was made, worth a deliberate re-check on a real trigger (here, an actual utilization threshold, not a calendar date) rather than migrated preemptively or left unexamined indefinitely.
5. A team new to GCP proposes running every workload — spiky HTTP APIs, event-driven processing, and steady batch jobs alike — on GKE Standard, reasoning that "one platform is simpler to operate than three." What's the flaw in that reasoning, given this chapter's decision framework?
The flaw is optimizing for platform-count simplicity instead of workload-fit, the same trade-off this chapter's decision table explicitly weighs against: a spiky HTTP API loses Cloud Run's scale-to-zero cost efficiency and instant traffic-splitting rollback running on GKE instead; event-driven processing loses Eventarc's purpose-built 90+ event-source integration; and a steady batch job likely runs more expensively on GKE than on Part 4's Compute Engine with Spot capacity. "Fewer platforms" is a real, legitimate operational value — but Meridian's own three-way split (Cloud Run, GKE, Compute Engine) reflects the conclusion that workload-fit costs (both financial and operational, like the cold-start problem this chapter documented) outweigh the benefit of artificial platform consolidation, for a team that already has genuine reason to run all three.
Summary and What's Next#
This chapter covered the middle and far end of the shared-responsibility spectrum: GKE's Autopilot/Standard/hybrid modes and when each fits, deploying and scaling workloads on GKE, Cloud Run's revision-and-traffic-splitting model that makes canary deploys and instant rollbacks natural rather than exceptional, Cloud Run functions and Eventarc for event-driven work, and — most importantly for real design work — a concrete decision framework for choosing between GKE, Cloud Run, and Part 4's Compute Engine for a new workload. Meridian's three-way platform split (shipment-api on Cloud Run, route-optimizer on GKE, GPS-ingestion on Compute Engine) demonstrates that the right answer is workload-specific, not a single platform standardized across an entire company.
The specific techniques worth carrying forward: default to the platform offering the most managed simplicity that still meets the workload's real requirements, treat cold starts and per-pod billing as real design inputs rather than afterthoughts, and use traffic splitting for every meaningful deploy rather than an all-or-nothing cutover.
Part 6 moves to data: Cloud Storage, the managed database options (Cloud SQL, Firestore, Spanner, BigQuery, and more), and how Meridian's shipment-api and route-optimizer actually persist and query the data these compute platforms process.