# Interview Questions: GCP DevOps & CI/CD Platform

# Part 1 Questions: Designing a DevOps Organization

## Conceptual

### 1. What's the core structural difference between what ACE tests and what PCDE tests?
ACE certifies you can operate individual GCP resources correctly. PCDE certifies you can design the system — the pipeline, the delivery process — that deploys those resources safely and repeatedly, at scale, without a human approving every change.

### 2. Why does a dedicated tooling project scale better than duplicating Cloud Build/Artifact Registry into every environment project?
A dedicated tooling project centralizes pipeline security review and maintenance to one place. Duplicating pipeline tooling per environment means a security fix has to be applied N times, and there's no guarantee two teams' "identical" pipelines actually stay in sync.

### 3. Why should environment-specific org policy differ between dev and prod rather than being identical?
Each environment has a different real risk tolerance — dev needs to move fast and tolerate experimentation, prod needs to fail closed. Identical policy across environments either over-restricts dev (slowing iteration for no safety benefit) or under-restricts prod (accepting real risk for no development-speed benefit).

### 4. Name three concrete ways CI/CD pipeline data carries data-residency obligations beyond the application's own data.
Build logs and source snapshots (which can contain PII if source includes copied test fixtures), container images (if they embed regional data), and Secret Manager/Cloud KMS key locations (which default to multi-region unless explicitly constrained).

### 5. What's the actual difference between Cloud Foundation Toolkit and Fabric FAST?
CFT is a library of individually reusable Terraform modules. Fabric FAST is a complete, opinionated multi-stage Terraform pipeline built from Google's Professional Services experience, organizing bootstrap into staged root modules with defined handoff contracts between them.

## Applied / Scenario

### 6. A pipeline service account was granted `roles/editor` "temporarily" during a migration, and four months later nobody wants to revoke it. What's the actual root cause of this situation, not just the symptom?
The immediate cause is scope creep in a temporary grant; the underlying root cause is that nobody defined the pipeline's actual required permission set before granting broad access, so there was no baseline to audit the grant against once it quietly became permanent.

### 7. A customer contract requires all resources to stay in `europe-west4`. What GCP mechanism enforces this, and at what level should it be applied?
A resource-location org policy constraint (`constraints/gcp.resourceLocations`), applied at the folder level covering that customer's dedicated project(s) — folder-level ensures any future project created under it inherits the constraint automatically.

### 8. Why is granting `roles/owner` to a deploy service account "to avoid repeatedly adding IAM bindings" the wrong fix for that friction?
`roles/owner` grants IAM policy changes, resource deletion, and data access far beyond deploy needs. The right fix for repeated-binding friction is defining the deployer's actual required permission set once (a custom role or a fixed list of predefined roles) and reapplying that same defined set to every new environment.

# Part 2 Questions: Infrastructure as Code & GitOps

## Conceptual

### 9. What specific risk does local Terraform state introduce once more than one person or process needs to run `apply`?
No locking and no sharing — a second `apply` from stale local state can propose to recreate or corrupt resources the first apply already created or modified, because neither state file has any record of the other's changes.

### 10. What does Infrastructure Manager actually manage that self-run Terraform doesn't?
Where `terraform plan`/`apply` runs and who operates that execution environment — Infra Manager runs Terraform inside a Google-managed environment with a Google-managed state backend, removing the need to operate your own CI runner and bootstrap your own state bucket.

### 11. Why must Terraform and Config Connector never manage the same live GCP resource?
Both reconcilers assume sole ownership of what they manage. Two controllers managing one resource produces flapping — each reverts the other's most recent change on its next reconciliation loop.

### 12. State the core GitOps rule in one sentence.
Git is the single source of truth for desired state, and an automated process — never a human running a command from a laptop — reconciles the live system to match whatever Git currently says.

### 13. What changed about Config Sync's licensing/tiering as of 2026, and why does it matter for older reference material?
Config Sync, Policy Controller, Fleet APIs, and Connect Gateway moved from a paid Anthos Enterprise tier into base GKE at no separate license cost — older material describing these as Anthos-exclusive enterprise features should be verified against current docs rather than trusted as-is.

## Applied / Scenario

### 14. A GKE cluster's live config has silently drifted from its Config Sync source repo, with no alert ever firing. What's the likely root cause?
A persistent `SyncError` went untriaged — Config Sync stops reconciling on a failure until the underlying issue is fixed, and every Git commit after that point silently never reaches the cluster until someone notices and fixes the sync error.

### 15. A team wants a new microservice's dedicated Pub/Sub topic reviewed in the same pull request as its Kubernetes Deployment, by the same reviewer, using the same tooling. Which tool from this chapter fits, and why not the alternative?
Config Connector — declaring the topic as a Kubernetes manifest keeps it in the same repo and review workflow as the Deployment YAML. Terraform would work technically but splits one logical change across two separate review processes and tools.

# Part 3 Questions: Continuous Integration with Cloud Build

## Conceptual

### 16. What is the actual scope of what Cloud Build does, and what does it explicitly NOT do?
Cloud Build takes source and turns it into a tested, versioned, packaged artifact in Artifact Registry — the "CI" half of CI/CD. It does not deploy anything; that's Cloud Deploy's job.

### 17. What's the difference between a built-in and a user-defined Cloud Build substitution, and how do you tell them apart in the YAML?
Built-in substitutions (`$PROJECT_ID`, `$SHORT_SHA`, `$BRANCH_NAME`) are filled in automatically from the triggering event with no declaration needed. User-defined substitutions must start with an underscore (`$_REGION`) and are declared explicitly in the `substitutions:` block or per-trigger.

### 18. Why shouldn't a container image be tagged `latest` in a CI pipeline?
It gives you no way to determine which specific commit is actually deployed and running — a critical gap during incident response. Tag with `$SHORT_SHA`/`$COMMIT_SHA` instead.

### 19. What's the key functional difference between a standard, a remote, and a virtual Artifact Registry repository?
Standard stores your own pushed artifacts. Remote proxies and caches an upstream public registry. Virtual unifies several standard/remote repos behind one URL with priority ordering. Vulnerability scanning does not run inside a virtual repository itself — only in the underlying repos it unifies.

### 20. When is a Cloud Build private pool required instead of the default shared pool?
Whenever a build step needs to reach a private-network resource — a Cloud SQL instance on a private IP, an on-prem system, or anything inside a VPC Service Controls perimeter — since the default shared pool has no route into any customer VPC.

## Applied / Scenario

### 21. A trigger fires on push to any branch with no filter, and a contractor's forty commits to an experimental branch in one afternoon burn through the monthly Cloud Build budget. What's the fix?
Add a branch-name regex filter (e.g., `^main$`) to push triggers that build and push artifacts, and use pull-request triggers — which can run tests without pushing anything — for feature-branch coverage.

### 22. Why can steps with no explicit `waitFor` run in parallel, and why is this a real correctness risk, not just a style issue?
Cloud Build does not guarantee sequential execution by list order alone — only explicit `waitFor` dependencies enforce ordering. If a later step depends on an earlier step's output and both run concurrently, the build can intermittently fail or succeed against a stale artifact.

### 23. A monorepo has one push trigger covering the whole repository, and every commit rebuilds every service regardless of which directory it touched. What's the fix?
Add `includedFiles`/`ignoredFiles` path filters to each service's own trigger, scoped to that service's directory (and any genuinely shared libraries it depends on) — a commit that doesn't touch a matching path never fires that trigger.

# Part 4 Questions: Continuous Delivery with Cloud Deploy

## Conceptual

### 24. What are the four core Cloud Deploy objects, and what does each declare?
`DeliveryPipeline` declares the ordered sequence of environments a release moves through. `Target` declares one environment's connection and promotion rules. `Release` is one immutable, versioned snapshot being promoted. `Rollout` is one specific attempt to deploy a Release to a Target.

### 25. What role does Skaffold play underneath Cloud Deploy?
Cloud Deploy delegates manifest rendering and applying to Skaffold — calling `skaffold render` to produce final environment-specific YAML (from Kustomize, Helm, or plain manifests) and `skaffold apply` to apply it to the target cluster.

### 26. What's the functional difference between canary and blue/green deployment strategies?
Canary gradually shifts a defined percentage of traffic to the new version with verification checkpoints between steps. Blue/green keeps both versions fully deployed and cuts traffic over in one atomic step rather than gradually.

### 27. What does `requireApproval: true` actually gate, and who can act on it?
It pauses a rollout after rendering, before any change reaches the cluster, until someone holding `roles/clouddeploy.approver` on that target explicitly approves it.

### 28. What happens if a `repairRolloutRule`'s configured retries are all exhausted with no explicit `rollback` block defined?
Cloud Deploy's default fallback behavior creates a new rollout to roll back to the most recently successful release on that target — rollback still happens by default, but the exact behavior of that fallback rollout is less explicit than a deliberately declared rollback rule.

## Applied / Scenario

### 29. A production target uses the standard strategy with no approval gate, and a bad release caused a full-traffic outage. What two independent fixes address this, and why are both worth making?
Add `requireApproval: true` (a human catches problems before rollout starts) and switch to canary with `verify: true` (limits exposure and catches problems that only manifest under real traffic). They're complementary — approval doesn't guarantee post-deploy health, and canary alone doesn't stop an ill-considered change from starting its rollout.

### 30. A `repairRolloutRule` has zero configured retries, and a single transient two-second DNS blip triggered a full automatic rollback of an otherwise healthy release. What's the fix?
Configure a small number of retries with exponential backoff before rollback — this distinguishes a genuinely broken release from a transient failure class that resolves on its own, without weakening the safety the rollback rule provides.

# Part 5 Questions: Managing Environments, Configuration & Secrets

## Conceptual

### 31. What's the core distinction between Secret Manager and Parameter Manager?
Secret Manager stores genuinely sensitive credentials (API keys, passwords, private keys) with strict access control and rotation. Parameter Manager manages configuration data — sensitive or not — like connection strings and feature flags, with built-in JSON/YAML format validation, and can reference a Secret Manager secret inline for its sensitive fields.

### 32. Why is an `ENV`/`--build-arg` value in a Dockerfile still build-time secret injection even if the running container never prints it?
Docker build arguments and ENV values are recorded in the image's build history and layer metadata, recoverable via `docker history` by anyone who can pull the image — independent of what the running container ever visibly does with the value.

### 33. Why doesn't creating a new Secret Manager secret version automatically fix a leaked credential for an already-running process?
A long-running process that cached the secret's value in memory at startup has no mechanism to see a newer version until it explicitly re-reads — creating a new version changes what Secret Manager serves, not what's already held in memory.

### 34. What real problem do ephemeral environments solve that a single persistent staging environment can't?
Two feature branches needing to modify the same database schema (or otherwise conflicting state) can't safely share one persistent staging environment at the same time — each gets its own throwaway environment with no conflict.

### 35. What changed about GKE fleet-management features' licensing as of 2026?
The Fleet API, Config Sync, Policy Controller, and Connect Gateway — previously requiring the paid Anthos/GKE Enterprise tier — moved into base GKE at no separate license cost.

## Applied / Scenario

### 36. A service's DB host/port and its DB password both currently live in the same Secret Manager secret as one blob. What's the mismatch, and what's the better structure?
The host/port is non-sensitive configuration, not a credential — storing it in Secret Manager adds unnecessary audit/access overhead for a value that isn't sensitive. Better structure: host/port in Parameter Manager, password in Secret Manager, with the Parameter Manager entry referencing the Secret Manager secret for its one sensitive field.

### 37. A team sets a maintenance exclusion for an upcoming high-traffic period, but only after a GKE upgrade has already started causing disruption. Why doesn't this help?
Maintenance exclusions block *future* automated upgrade attempts during the excluded window — they don't retroactively pause or reverse an upgrade already in progress. Exclusions must be set before the high-risk period begins.

# Part 6 Questions: Securing the Deployment Pipeline & Dev Environments

## Conceptual

### 38. What's the actual difference between what SLSA grades and what Artifact Analysis checks?
SLSA grades the build platform's own trustworthiness and tamper-resistance (the pipeline's integrity). Artifact Analysis scans a specific artifact for known vulnerabilities in its dependencies. A SLSA Level 3-compliant pipeline can still faithfully build an artifact with a genuine vulnerable dependency — the two are complementary, not overlapping.

### 39. What does Binary Authorization's `defaultAdmissionRule` control, and why is a permissive default dangerous?
It's the fallback rule applied to any cluster with no explicit `clusterAdmissionRules` entry. A permissive default (`ALWAYS_ALLOW`) means any cluster not explicitly locked down accepts any image with zero attestation requirement, regardless of how strict other clusters' explicit rules are.

### 40. What does the persistent-disk-backed `/home` versus ephemeral-image split in Cloud Workstations actually buy you?
Anything a developer needs to survive across sessions lives only in `/home` (persistent); anything in the container image is reproducible, auditable, and identical across every engineer using that image — the same stateless-container-plus-separate-volume trade-off application workloads already use.

### 41. What is Software Delivery Shield, structurally?
A bundled framing/name for the combination of source integrity, build provenance (SLSA), vulnerability scanning (Artifact Analysis), and deploy-time enforcement (Binary Authorization) — not a separate product with its own API distinct from configuring those individual controls.

## Applied / Scenario

### 42. A security review finds `meridian-staging` has no entry in `clusterAdmissionRules`, while `defaultAdmissionRule` is `ALWAYS_ALLOW`. What's the actual posture of `meridian-staging` right now?
Any image from any source can deploy there with zero attestation requirement, regardless of how strict other clusters' explicit rules are — it inherits the permissive default entirely.

### 43. A team believes "SLSA Level 3" means their application has no supply-chain vulnerabilities. How do you correct this?
SLSA Level 3 certifies the build platform's resistance to tampering and provenance forgery, not the absence of vulnerabilities in what it builds — a hardened, Level 3-compliant pipeline can still faithfully build and ship an artifact containing a real vulnerable dependency, which is Artifact Analysis's separate job to catch.

### 44. Developers are told to manually install their tools inside each Cloud Workstations session instead of using a custom image. What breaks, given the storage model?
Anything installed outside `/home` lives only in that session's ephemeral image layer and does not persist to the next session — every developer re-runs manual installs every session, and different developers likely drift to inconsistent tool versions, reintroducing the "works on my machine" inconsistency a shared custom image exists to prevent.

## Quick-Fire Recall

| Term/Question | Answer |
|---|---|
| Tooling project pattern | One project holds Cloud Build/Artifact Registry/Cloud Deploy; environment projects hold no pipeline machinery |
| CFT vs Fabric FAST | CFT = reusable module library; FAST = complete staged Terraform bootstrap pipeline |
| GCS Terraform backend | Remote state with real locking, replacing risky local `.tfstate` |
| Config Connector | Kubernetes-native GCP resource management via CRDs and in-cluster reconciliation |
| GitOps core rule | Git is the source of truth; automation reconciles live state to match it, no human-run apply |
| Config Sync 2026 tiering | Moved from paid Anthos tier into base GKE |
| Built-in vs user-defined substitution | Built-in: `$PROJECT_ID`/`$SHORT_SHA`, automatic. User-defined: `$_NAME`, underscore-prefixed, declared |
| Image tagging rule | Tag with `$SHORT_SHA`, never `latest` |
| Virtual repo scanning limitation | Artifact Analysis does not scan inside virtual repositories |
| Private pool use case | Reaching private VPC resources (Cloud SQL private IP, on-prem, VPC-SC perimeter) |
| `waitFor` | Explicit step dependency; without it, steps can run in parallel regardless of list order |
| Monorepo trigger fix | `includedFiles`/`ignoredFiles` path filters per service |
| DeliveryPipeline / Target / Release / Rollout | Stage sequence / one environment / one immutable snapshot / one deploy attempt |
| Canary vs blue/green | Canary: gradual traffic shift with verification steps. Blue/green: atomic full cutover |
| `requireApproval` | Gates a rollout on a human holding `roles/clouddeploy.approver` |
| Automation `repairRolloutRule` | Defines retry count/backoff before automatic rollback on verification failure |
| Secret Manager vs Parameter Manager | Credentials vs configuration; Parameter Manager can reference a Secret Manager secret |
| Build-time vs runtime secret injection | Build-time bakes into the image (recoverable via `docker history`); runtime fetches at startup via Workload Identity |
| Ephemeral environment risk | No TTL means it becomes a permanent environment with a misleading name |
| GKE release channels | Rapid (newest), Regular (balanced), Stable (most conservative) |
| Maintenance exclusion | Blocks future upgrades for up to 90 days; doesn't undo an upgrade already in progress |
| SLSA levels | L1: documented + provenance. L2: hosted, tamper-evident. L3: hardened, forgery-resistant platform |
| Binary Authorization default posture | Should be `ALWAYS_DENY`; a permissive default silently exempts unlisted clusters |
| Software Delivery Shield | Bundled name for source integrity + provenance + scanning + enforcement, not a separate product |
| Cloud Workstations storage | Only `/home` is persistent; everything else comes fresh from the container image every session |