# GCP DevOps & CI/CD Platform — Part 6: Securing the Deployment Pipeline & Dev Environments

> **Series:** GCP DevOps & CI/CD Platform (6 of 6) — aligned to Professional Cloud DevOps Engineer (PCDE) exam sections 1-2
> **Part 1:** `01-designing-a-devops-organization.md` — Designing a DevOps Organization
> **Part 2:** `02-infrastructure-as-code-and-gitops.md` — Infrastructure as Code & GitOps
> **Part 3:** `03-continuous-integration-with-cloud-build.md` — Continuous Integration with Cloud Build
> **Part 4:** `04-continuous-delivery-with-cloud-deploy.md` — Continuous Delivery with Cloud Deploy
> **Part 5:** `05-managing-environments-and-secrets.md` — Managing Environments, Configuration & Secrets
> **Part 6:** This file — Securing the Deployment Pipeline & Dev Environments
> **Questions:** `questions.md`

## Table of Contents

1. [Why the Pipeline Itself Is an Attack Surface](#why-the-pipeline-itself-is-an-attack-surface)
2. [Artifact Analysis: Scanning What the Pipeline Produces](#artifact-analysis-scanning-what-the-pipeline-produces)
3. [The SLSA Framework: Grading a Pipeline's Own Trustworthiness](#the-slsa-framework-grading-a-pipelines-own-trustworthiness)
4. [Binary Authorization: Enforcing What's Allowed to Deploy](#binary-authorization-enforcing-whats-allowed-to-deploy)
5. [Software Delivery Shield: The Bundled Picture](#software-delivery-shield-the-bundled-picture)
6. [IAM Policies Based on Environment](#iam-policies-based-on-environment)
7. [Cloud Workstations: Secure, Standardized Development Environments](#cloud-workstations-secure-standardized-development-environments)
8. [Bootstrapping Environments With Required Tooling](#bootstrapping-environments-with-required-tooling)
9. [AI-Assisted Development and Operations](#ai-assisted-development-and-operations)
10. [A Full Worked Example: Meridian's End-to-End Supply Chain Gate](#a-full-worked-example-meridians-end-to-end-supply-chain-gate)
11. [Common Mistakes and Interview Traps](#common-mistakes-and-interview-traps)
12. [Worked Practice Problems](#worked-practice-problems)
13. [Summary and What's Next](#summary-and-whats-next)

## Why the Pipeline Itself Is an Attack Surface

Every prior chapter in this course treated the CI/CD pipeline as the trusted mechanism delivering code safely — this final chapter asks the harder question the exam guide's "securing the deployment pipeline" section (2.4) is really about: **what stops the pipeline itself from becoming the attack?** A pipeline with unrestricted deploy permissions, no verification of what it's actually shipping, and no record of how an artifact was built is a single high-value target — compromise the pipeline once, and every environment it deploys to is compromised too, often with the legitimate-looking audit trail of an authorized deployment rather than an obvious intrusion.

This is the real-world category of incident behind the software supply-chain attacks that made the SLSA framework and Binary Authorization mainstream priorities industry-wide, not just a GCP-specific concern: a compromised build step, a tampered dependency, or a stolen deploy credential can all produce an artifact that looks completely legitimate to every check this course has built so far, unless the pipeline is specifically hardened against exactly that.

```mermaid
flowchart TD
    Source["Source repo"] --> Build["Cloud Build (Part 3)"]
    Build --> Registry["Artifact Registry"]
    Registry --> Deploy["Cloud Deploy (Part 4)"]

    Attack1["Compromised dependency"] -.->|"injects malicious code"| Build
    Attack2["Stolen deploy credential"] -.->|"bypasses pipeline entirely"| Deploy
    Attack3["Tampered image after build"] -.->|"swaps digest before deploy"| Registry

    classDef normal fill:#e5f0fa,stroke:#1d6fb8,color:#10161c
    classDef attack fill:#fbe8e6,stroke:#b3261e,color:#10161c
    class Source,Build,Registry,Deploy normal
    class Attack1,Attack2,Attack3 attack
```

**What to notice**: each attack path targets a different stage, which is exactly why this chapter's defenses (scanning, provenance, attestation-gated deploy) are layered rather than a single control — a defense that only covers the build step does nothing against a stolen deploy credential bypassing the build entirely.

## Artifact Analysis: Scanning What the Pipeline Produces

Part 3 introduced Artifact Analysis briefly as the mechanism scanning images pushed to Artifact Registry; here's the full picture. Artifact Analysis automatically scans container images for known CVEs across OS packages and, as of recent GA coverage, application-level dependencies too (Python and Node.js package vulnerabilities, not just OS-layer ones) — a real, worth-knowing expansion beyond "just scans the base OS image."

```bash
# Query scan results for the exact image Part 3's build produced --
# this is the check a Binary Authorization attestor (next section)
# would gate deployment on
gcloud artifacts docker images list-vulnerabilities \
  us-central1-docker.pkg.dev/meridian-cicd/meridian-images/shipment-api@sha256:abc123... \
  --project=meridian-cicd \
  --format="table(vulnerability.effectiveSeverity, vulnerability.shortDescription)"
```

> [!IMPORTANT]
> Add a Cloud Build step that fails the build outright on any `CRITICAL` (and typically `HIGH`) severity finding, placed *before* the push-to-registry step from Part 3's fail-fast ordering — scanning that only reports findings after an image is already pushed and potentially already deployed is a dashboard, not a gate. The distinction matters: a gate stops the bad artifact from ever leaving the pipeline; a dashboard just tells you about it after the fact.

## The SLSA Framework: Grading a Pipeline's Own Trustworthiness

**SLSA** (Supply-chain Levels for Software Artifacts, pronounced "salsa") is an industry framework — originally derived from Google's own internal "Binary Authorization for Borg" system, used company-wide for over a decade before being generalized and open-sourced — that grades a build pipeline's own trustworthiness on an increasing scale, rather than grading any single artifact.

| SLSA level | What it requires | What it defends against |
|---|---|---|
| Level 1 | The build process is documented and produces provenance metadata | Establishes a baseline record of how an artifact was built |
| Level 2 | Provenance is generated by a hosted build service (not a developer's laptop), and is tamper-evident | A developer silently substituting a different artifact than what CI actually produced |
| Level 3 | The build platform itself is hardened — isolated, ephemeral build environments, provenance that can't be forged even by someone with some access to the build system | An attacker with partial access to the build infrastructure forging a legitimate-looking provenance record |

Cloud Build provides SLSA Level 3-capable provenance generation out of the box for supported build configurations — isolated, ephemeral build environments (each build gets a clean environment, never reused across builds) plus cryptographically signed provenance attesting to exactly which source commit, which build steps, and which build service produced a given image digest.

```mermaid
flowchart LR
    L1["Level 1<br/>Documented + provenance exists"] --> L2["Level 2<br/>Hosted, tamper-evident provenance"]
    L2 --> L3["Level 3<br/>Hardened, forgery-resistant<br/>build platform"]

    classDef base fill:#e5f0fa,stroke:#1d6fb8,color:#10161c
    classDef mid fill:#fbeee0,stroke:#b8650f,color:#10161c
    classDef strong fill:#e5f5ea,stroke:#1f8a4c,color:#10161c
    class L1 base
    class L2 mid
    class L3 strong
```

💡 **The transferable insight**: SLSA's escalating levels mirror the same trust-tiering logic as a financial audit's control maturity model — Level 1 is "a record exists," Level 2 is "the record can't be quietly altered after the fact by an ordinary user," Level 3 is "the record can't be forged even by someone with elevated access to the system producing it." Each level closes a progressively more sophisticated threat, not just "more of the same" documentation.

## Binary Authorization: Enforcing What's Allowed to Deploy

SLSA provenance and vulnerability scan results are only useful if something actually *enforces* them at deploy time — that's Binary Authorization's job. It's a deploy-time admission controller for GKE (and Cloud Run) that blocks any image lacking the required **attestations** — cryptographically signed statements from a trusted **attestor** confirming a specific check passed (the vulnerability scan came back clean, the image was built by the expected Cloud Build pipeline, a QA sign-off happened).

```yaml
# Binary Authorization policy -- requires an attestation from BOTH
# the automated build attestor AND a QA attestor before any image
# can deploy to Meridian's production cluster
name: projects/meridian-prod/policy
globalPolicyEvaluationMode: ENABLE
defaultAdmissionRule:
  evaluationMode: ALWAYS_DENY
  enforcementMode: ENFORCED_BLOCK_AND_AUDIT_LOG
clusterAdmissionRules:
  us-central1-a.meridian-prod:
    evaluationMode: REQUIRE_ATTESTATION
    enforcementMode: ENFORCED_BLOCK_AND_AUDIT_LOG
    requireAttestationsBy:
      - projects/meridian-cicd/attestors/vulnerability-scan-passed
      - projects/meridian-cicd/attestors/qa-signoff
```

**What to notice about the `defaultAdmissionRule`**: it's set to `ALWAYS_DENY` — a deliberate default-deny posture. Only the explicitly listed cluster (`meridian-prod`, in this case) has a rule permitting deployment at all, and even that requires both attestations. Any image, from any source, targeting any cluster not explicitly covered by a permissive rule is blocked by default.

```bash
# The build pipeline itself creates the attestation, immediately
# after the vulnerability scan step from earlier in this chapter
# passes cleanly -- this is what a Cloud Build step actually runs
gcloud container binauthz attestations sign-and-create \
  --project=meridian-cicd \
  --artifact-url="us-central1-docker.pkg.dev/meridian-cicd/meridian-images/shipment-api@sha256:abc123..." \
  --attestor=vulnerability-scan-passed \
  --attestor-project=meridian-cicd \
  --keyversion=projects/meridian-cicd/locations/us-central1/keyRings/binauthz/cryptoKeys/attestor-key/cryptoKeyVersions/1
```

> [!WARNING]
> Binary Authorization only gates what it's actually enabled on — a cluster with no `clusterAdmissionRules` entry falls through to `defaultAdmissionRule`, and a *permissive* default rule left in place "temporarily" during initial rollout defeats the entire control for every cluster that hasn't been explicitly locked down yet. Confirm the default is `ALWAYS_DENY` (or an equivalently strict rule) before considering Binary Authorization actually enforced anywhere.

## Software Delivery Shield: The Bundled Picture

Every control in this chapter so far — Artifact Analysis, SLSA-aligned build provenance, and Binary Authorization — is also sold and documented together under Google's **Software Delivery Shield** umbrella, a bundled framing of end-to-end supply-chain security across the software lifecycle rather than a single new product to learn. It's worth knowing the name and the framing (the exam guide's "software supply chain security" phrasing points at exactly this), but the individual controls are what actually get configured — there's no separate "Software Delivery Shield" API or console page distinct from turning on the pieces this chapter already covered.

```mermaid
flowchart LR
    subgraph shield["Software Delivery Shield (the bundled framing)"]
        direction LR
        Source["Source integrity"] --> BuildP["Build provenance<br/>(SLSA, Cloud Build)"]
        BuildP --> Scan["Vulnerability scanning<br/>(Artifact Analysis)"]
        Scan --> Enforce["Deploy-time enforcement<br/>(Binary Authorization)"]
    end

    classDef stage fill:#e5f0fa,stroke:#1d6fb8,color:#10161c
    class Source,BuildP,Scan,Enforce stage
```

> [!NOTE]
> Don't over-index on the marketing name for the exam or in a design conversation — describe the actual mechanism ("we require a vulnerability-scan attestation before deploy") rather than "we use Software Delivery Shield," which says nothing about what's actually configured. This mirrors the same "concepts over product-name trivia" caution from this course's third-party tooling section in Part 3.

## IAM Policies Based on Environment

This closes a loop the whole course has been building toward: Part 1 established cross-project IAM scoped per target, and this section names the principle explicitly as an exam topic in its own right — **IAM grants should differ by environment, not just by role**. The same "deployer" identity legitimately needs different permissions in `meridian-dev` versus `meridian-prod`, and a security review should be able to answer "what can this pipeline actually do in production" without cross-referencing every environment's grants by hand.

| Environment | Deploy permission scope | Attestation requirement | Who can approve |
|---|---|---|---|
| `meridian-dev` | Broad within the namespace — fast iteration | None | Auto-promoted, no approval |
| `meridian-staging` | Namespace-scoped, matches prod's shape for realistic testing | Vulnerability scan only | Manual trigger, no formal approval |
| `meridian-prod` | Narrowest — exact resources only | Both vulnerability scan and QA sign-off | `roles/clouddeploy.approver` holders only |

This table is the same shape as Part 1's environment-policy table and Part 4's approval-gate discussion — by this point in the course, the pattern should feel familiar: **every chapter's security control tightens in the same direction, from dev toward prod, and none of them exist as a one-off decision made in isolation from the others.**

## Cloud Workstations: Secure, Standardized Development Environments

The exam guide's "enabling secure cloud development environments" (1.5) shifts focus from the pipeline to where code is actually *written*. Cloud Workstations provides browser- or IDE-accessible, fully managed development environments running on ephemeral Compute Engine VMs, with a persistent disk backing only the `/home` directory — everything else comes from the workstation's container image and is rebuilt fresh on every session start.

```mermaid
flowchart TB
    subgraph vm["Ephemeral Compute Engine VM"]
        Image["Container image<br/>(tools, system packages)"]
        Home["/home<br/>(persistent disk, survives restarts)"]
    end
    Dev["Engineer connects<br/>via browser or IDE"] --> vm
    Image -.->|"rebuilt fresh<br/>every session"| vm
    Home -.->|"user files, settings<br/>persist across sessions"| vm

    classDef ephemeral fill:#fbeee0,stroke:#b8650f,color:#10161c
    classDef persistent fill:#e5f5ea,stroke:#1f8a4c,color:#10161c
    class Image ephemeral
    class Home persistent
```

**What to notice**: this ephemeral-image/persistent-home split is the same underlying trade-off as a stateless application container backed by a separate persistent volume — anything a developer needs to survive isn't in the image at all, and anything in the image is reproducible, auditable, and identical for every engineer using it.

> [!TIP]
> **Best Practice**: build custom Cloud Workstations images from Google's own preconfigured base images (which already ship VS Code or a JetBrains IDE), adding only what the team's actual stack needs on top — a pinned Go/Node/Python toolchain version, internal CLI tools like Meridian's own `merictl` from Part 1. Run Artifact Analysis against the custom image the same way Part 3's pipeline scans application images; a workstation image with an unpatched base OS is just as real a risk surface as a production container.

## Bootstrapping Environments With Required Tooling

A custom Cloud Workstations image bakes in tools available to every session from the moment it starts, using the same base-image-plus-startup-script convention as most container customization:

```dockerfile
# Custom Cloud Workstations image -- extends Google's base VS Code
# image, adding Meridian's pinned toolchain and internal CLI
FROM us-central1-docker.pkg.dev/cloud-workstations-images/predefined/code-oss:latest

# Pin exact versions -- an unpinned toolchain drifts silently across
# engineers' sessions, the same "one image, many consumers" principle
# from Part 1's Artifact Registry discussion, applied to dev tooling
RUN curl -fsSL https://go.dev/dl/go1.24.0.linux-amd64.tar.gz | tar -C /usr/local -xz
ENV PATH="/usr/local/go/bin:${PATH}"

# Install the internal merictl CLI from Part 1's automation chapter,
# from Meridian's own Artifact Registry generic repository
COPY --from=us-central1-docker.pkg.dev/meridian-cicd/meridian-tools/merictl:latest /merictl /usr/local/bin/merictl

# Startup scripts run in lexicographical order on every session start
COPY startup/configure-gcloud.sh /etc/workstation-startup.d/100-configure-gcloud.sh
```

🧪 **Hands-on checkpoint**: set up a scheduled Cloud Build trigger (Part 3's tooling, applied to workstation images now instead of application images) that rebuilds this image whenever Google publishes a new base image update, and confirm the rebuild pipeline re-runs the same Artifact Analysis scan every application image goes through — a stale, unpatched workstation base image is a genuine, easy-to-overlook drift risk otherwise.

## AI-Assisted Development and Operations

The exam guide explicitly names "leveraging AI to assist with development and operations (e.g., Gemini Code Assist, Gemini Cloud Assist, Gemini CLI)" under secure development environments — a real, current reflection of how GCP's own developer tooling has evolved, not a tangential mention. **Gemini Code Assist** provides in-IDE code completion and chat grounded in a codebase's actual context, available inside Cloud Workstations sessions. **Gemini Cloud Assist** answers operational questions across GCP's own console and APIs — surfacing a likely root cause for a failing deploy, or summarizing what changed in a project recently. **Gemini CLI** brings the same assistance to a terminal-first workflow, scriptable alongside the `gcloud`/`kubectl` commands this whole course has used directly.

> [!NOTE]
> These tools assist a human's judgment inside the workflows this course has built — they don't replace the actual controls (Binary Authorization's attestation requirement, the approval gate on `meridian-prod`) covering *what's allowed to happen*. Treat AI assistance the same way you'd treat a very well-read colleague's suggestion: useful input, not a substitute for the pipeline's own enforced policy.

## A Full Worked Example: Meridian's End-to-End Supply Chain Gate

Bringing every chapter of this course together into the one pipeline Meridian actually runs today — this is the complete picture Part 1's tooling-project design was always building toward.

```mermaid
sequenceDiagram
    participant Dev as Engineer<br/>(Cloud Workstations)
    participant CB as Cloud Build
    participant AA as Artifact Analysis
    participant BA as Binary Authorization
    participant CDep as Cloud Deploy

    Dev->>CB: Push commit (from a hardened,<br/>scanned workstation image)
    CB->>CB: Lint, unit test, build, integration test<br/>(Part 3's fail-fast order)
    CB->>AA: Scan built image
    AA-->>CB: No critical/high findings
    CB->>CB: Sign-and-create attestation<br/>(vulnerability-scan-passed)
    CB->>CDep: Create release
    CDep->>CDep: Auto-promote to staging (Part 4)
    Note over CDep: Manual QA sign-off creates<br/>the second attestation
    CDep->>BA: Attempt deploy to meridian-prod
    BA->>BA: Check BOTH attestations present
    BA-->>CDep: Authorized -- deploy proceeds
    CDep->>CDep: Canary rollout, approval gate,<br/>automated rollback (Part 4)
```

**What to notice**: this single diagram is every chapter of this course in sequence — a hardened dev environment (this chapter), tested and scanned CI (Part 3), gated and staged CD (Part 4), environment-appropriate configuration (Part 5), and a supply-chain enforcement point that would block the deploy entirely if either attestation were missing (this chapter). No single control here is sufficient alone; together they're the actual answer to "can you build the pipeline that deploys hundreds of releases correctly, every day, without a human approving each one" from Part 1's framing of what PCDE tests.

## Common Mistakes and Interview Traps

| Mistake | Why it's wrong | What to say/do instead |
|---|---|---|
| Treating vulnerability scanning as a report, not a gate | Findings surfacing after an image is already pushed/deployed don't prevent the risk | Fail the build on critical/high findings before the push step |
| Leaving Binary Authorization's default admission rule permissive | Any cluster without an explicit strict rule inherits the permissive default | Set `defaultAdmissionRule` to deny, add explicit permissive rules only where deliberately intended |
| Assuming SLSA levels grade an artifact | SLSA grades the *build platform's* trustworthiness, not any single image | Reason about SLSA level as a property of the pipeline, not a per-artifact label |
| Using identical IAM grants for a deploy service account across every environment | Ignores the real different-risk-tolerance-per-environment principle from Part 1 | Scope grants narrowest in prod, looser in dev, matched to each environment's actual risk |
| Baking developer tooling into a workstation image and never rebuilding it | An unpatched base image is as real a risk as an unpatched production container | Scan workstation images the same as application images; automate rebuilds on base-image updates |
| Treating Gemini Code/Cloud Assist output as authoritative without verification | AI assistance is a suggestion, not an enforced control | Keep the actual gates (attestation, approval) as the real authority; use AI tools to assist judgment |

## Worked Practice Problems

**Problem 1**: A security review finds that `meridian-staging`'s GKE cluster has no entry in Binary Authorization's `clusterAdmissionRules`, while `meridian-prod` has a strict `REQUIRE_ATTESTATION` rule. The `defaultAdmissionRule` is set to `ALWAYS_ALLOW`. What's the actual security posture of `meridian-staging` right now, and is this likely intentional?

*Answer*: Because `meridian-staging` has no explicit cluster rule, it falls through to the `defaultAdmissionRule` — which is `ALWAYS_ALLOW`, meaning *any* image from *any* source can deploy there with zero attestation requirement, regardless of how strict `meridian-prod`'s explicit rule is. This is almost certainly unintentional: a permissive default is a common artifact of initial rollout ("get prod locked down first"), and it should be tightened to at least require the vulnerability-scan attestation, with `meridian-prod`'s additional QA sign-off requirement layered on top for that stricter environment specifically.

**Problem 2**: A team asks whether achieving "SLSA Level 3" means their production application has no supply-chain vulnerabilities. How would you correct this understanding?

*Answer*: SLSA Level 3 certifies the *build platform's* resistance to tampering and provenance forgery — it says the pipeline that produced an artifact is hardened and its provenance can be trusted, not that the artifact itself is free of vulnerabilities. A perfectly SLSA Level 3-compliant pipeline can still faithfully build and ship an application with a genuine vulnerable dependency; that's Artifact Analysis's job to catch, a separate and complementary control, not something SLSA level itself guarantees away.

**Problem 3**: An engineer wants to skip building a custom Cloud Workstations image and instead have every developer manually install their own tools inside the ephemeral session each time it starts. What's the concrete downside of this approach given how Cloud Workstations' storage model works?

*Answer*: Anything installed outside `/home` during a session lives only in the ephemeral container image layer for that session — it does not persist to the next session, since only `/home` is backed by persistent disk. Every developer would have to re-run their manual install steps on every new session start, and different developers would likely end up with inconsistent tool versions, reintroducing exactly the "works on my machine" inconsistency a shared, versioned custom image exists to eliminate.

## Summary and What's Next

This final chapter of the course closed the loop this whole series has been building: Artifact Analysis and SLSA as the trust-and-verification layer over what the pipeline produces, Binary Authorization as the deploy-time enforcement point that actually acts on that trust, environment-scoped IAM as the same "tighten toward prod" principle threaded through every prior chapter, and Cloud Workstations as the hardened, standardized starting point the entire pipeline traces back to. Meridian's end-to-end supply chain gate — from a scanned workstation image through fail-fast CI, dual-attestation-gated CD, and environment-correct configuration — is the complete, real answer to the Friday-afternoon incident that opened Part 1.

This completes **GCP DevOps & CI/CD Platform**, covering PCDE exam sections 1-2 (bootstrapping an organization and building CI/CD pipelines) in full. The next course in this series, **[GCP SRE & Observability](/tutorials/gcp-sre-observability)**, picks up PCDE's remaining sections — applying site reliability engineering practices, observability and troubleshooting, and performance/cost optimization — completing the certification's full scope.
