Secret Manager & Cloud KMS

What it is and where it fits 🎯#

Secret Manager stores and versions small sensitive values, API keys, database passwords, TLS private keys, so an application never has one baked into an environment variable, a config file, or (worst of all) source control. Cloud KMS is one level lower: it manages cryptographic keys and performs encrypt/decrypt/sign operations without the key material ever leaving Google's infrastructure. The two are complementary, not competing: Secret Manager can use a customer-managed KMS key to encrypt secret data at rest (shown below), and an application that needs to encrypt its own data (not just retrieve a stored credential) reaches for KMS directly. This page assumes the project/IAM basics from page 01; every secret and key below is protected by ordinary IAM bindings, not a separate access-control system.

Core concepts: a secret's versions, and where the actual bytes live#

A secret is a named container with an IAM policy; the actual sensitive bytes live in numbered, immutable versions underneath it. Adding a new value creates a new version rather than overwriting the old one, and an application always requests either latest or a specific version number, never "the secret" as a single mutable value, this is what makes a credential rotation an additive, reversible operation (disable the old version if the new one turns out broken) instead of a destructive overwrite.

Creating secrets and adding versions#

echo -n "s3cr3t-db-password" | gcloud secrets create db-password --data-file=-
gcloud secrets create api-key --data-file=./api-key.txt --labels=team=payments,env=prod

echo -n "new-rotated-password" | gcloud secrets versions add db-password --data-file=-
gcloud secrets versions list db-password

--data-file=- reads from stdin, the pattern that avoids ever writing the secret value to a file on disk or leaving it visible in shell history the way passing it as a literal command-line argument would. secrets create with no data at all is also valid, creating an empty secret container that CI can add the first version to later, useful when the secret's value doesn't exist yet at provisioning time (a value generated by a separate bootstrap step, say).

Reading secret values#

gcloud secrets versions access latest --secret=db-password
gcloud secrets versions access 2 --secret=db-password

gcloud secrets versions access latest --secret=api-key --out-file=/tmp/api-key.txt

versions access latest resolves to whichever version is currently ENABLED and most recent, the pattern almost every application's startup code uses; pinning to a specific version number is for the rarer case of deliberately testing against a known-old value. versions access prints the raw secret bytes to stdout by default, worth piping directly into whatever consumes it (| psql, | docker login --password-stdin) rather than assigning it to a shell variable that then shows up in env output or shell history.

Access control: who can read a secret#

gcloud secrets add-iam-policy-binding db-password \
  --member="serviceAccount:api@my-project-id.iam.gserviceaccount.com" \
  --role="roles/secretmanager.secretAccessor"

gcloud secrets get-iam-policy db-password
gcloud secrets remove-iam-policy-binding db-password \
  --member="serviceAccount:old-service@my-project-id.iam.gserviceaccount.com" \
  --role="roles/secretmanager.secretAccessor"

roles/secretmanager.secretAccessor is deliberately the narrowest useful role, read-only, granted per secret rather than project-wide. Granting it at the project level instead of per secret (via gcloud projects add-iam-policy-binding on page 01) gives that identity read access to every secret in the project, almost never the intended scope, per-secret bindings shown here are the pattern to default to.

Rotation and expiry#

gcloud secrets create db-password \
  --replication-policy=automatic \
  --next-rotation-time=2026-10-01T00:00:00Z --rotation-period=2592000s \
  --topics=projects/my-project-id/topics/secret-rotation-alerts

gcloud secrets versions disable 1 --secret=db-password   # old version stays retrievable but not "latest"
gcloud secrets versions destroy 1 --secret=db-password    # irreversible — the bytes are actually gone

--rotation-period combined with --topics doesn't rotate the secret's value for you, Secret Manager has no built-in credential-generation logic, it publishes a Pub/Sub notification on schedule that a Cloud Function or Workflow (pages 05 and 10) subscribes to and actually performs the rotation against, whatever downstream system owns the credential.

Cloud KMS: keyrings and keys#

gcloud kms keyrings create app-keyring --location=us-central1
gcloud kms keys create app-encryption-key \
  --keyring=app-keyring --location=us-central1 \
  --purpose=encryption --rotation-period=7776000s --next-rotation-time=2026-12-01T00:00:00Z

gcloud kms keys list --keyring=app-keyring --location=us-central1
gcloud kms keys describe app-encryption-key --keyring=app-keyring --location=us-central1

A keyring is purely an organizational grouping, IAM permissions are typically granted at the keyring level so every key underneath inherits them, rather than managing access per individual key. --purpose is required and fixed at creation, encryption for symmetric encrypt/decrypt (the common case), asymmetric- signing or asymmetric-encryption for public/private key pairs, you cannot change a key's purpose after creation, a wrong choice here means creating a new key, not patching the existing one.

Customer-managed encryption keys (CMEK): using a KMS key to protect another service's data#

gcloud kms keys add-iam-policy-binding app-encryption-key \
  --keyring=app-keyring --location=us-central1 \
  --member="serviceAccount:service-<project-number>@gcp-sa-cloud-sql.iam.gserviceaccount.com" \
  --role="roles/cloudkms.cryptoKeyEncrypterDecrypter"

gcloud sql instances create cmek-protected-instance \
  --database-version=POSTGRES_16 --region=us-central1 --tier=db-custom-2-8192 \
  --disk-encryption-key=projects/my-project-id/locations/us-central1/keyRings/app-keyring/cryptoKeys/app-encryption-key

gcloud storage buckets create gs://cmek-protected-bucket \
  --location=us-central1 --default-encryption-key=projects/my-project-id/locations/us-central1/keyRings/app-keyring/cryptoKeys/app-encryption-key

By default, every GCP service already encrypts data at rest with a Google-managed key, CMEK doesn't add encryption where none existed, it swaps who controls the key, and therefore who can revoke access to the underlying data entirely by disabling the key. Every CMEK-consuming service (Cloud SQL, Cloud Storage, BigQuery, and others) needs its own dedicated service agent granted cryptoKeyEncrypterDecrypter on the key before the resource is created, shown here for Cloud SQL's service agent, the exact principal differs per service, gcloud services list --enabled combined with that service's own documentation is how to find the right one.

Encrypting and decrypting data directly#

gcloud kms encrypt \
  --key=app-encryption-key --keyring=app-keyring --location=us-central1 \
  --plaintext-file=config.json --ciphertext-file=config.json.enc

gcloud kms decrypt \
  --key=app-encryption-key --keyring=app-keyring --location=us-central1 \
  --ciphertext-file=config.json.enc --plaintext-file=config.json

The raw key material never leaves Google's KMS infrastructure for either operation, gcloud (and any client library) sends the plaintext/ciphertext to the KMS API and gets the result back, it never has direct access to the key bytes themselves, which is the entire security property KMS provides over, say, a key stored in an environment variable and used with a local crypto library. The 64KiB size limit on kms encrypt's plaintext file (from the tool's own help text) means this direct path is for small configuration blobs or, more commonly, envelope encryption, using KMS to encrypt a locally-generated data encryption key, then using that (unlimited-size) key to encrypt the actual payload.

Key rotation and IAM#

gcloud kms keys add-iam-policy-binding app-encryption-key \
  --keyring=app-keyring --location=us-central1 \
  --member="serviceAccount:api@my-project-id.iam.gserviceaccount.com" \
  --role="roles/cloudkms.cryptoKeyEncrypterDecrypter"

gcloud kms keys versions list --key=app-encryption-key --keyring=app-keyring --location=us-central1
gcloud kms keys versions disable 1 --key=app-encryption-key --keyring=app-keyring --location=us-central1

roles/cloudkms.cryptoKeyEncrypterDecrypter grants use of the key (encrypt/decrypt operations) without granting the ability to manage the key itself (rotate it, change its IAM policy), the same least-privilege split Secret Manager's secretAccessor role embodies. Like a secret, a key has numbered versions underneath it, --rotation-period automatically creates a new primary version on schedule, but never deletes old versions automatically, data encrypted under an older version still needs that version enabled to be decrypted later.

Real-world scenario: bootstrapping a database password with no human ever seeing it#

A new environment needs a database credential that's generated, stored, and consumed entirely by automation:

openssl rand -base64 32 | gcloud secrets create db-password-staging \
  --data-file=- --replication-policy=automatic

gcloud secrets add-iam-policy-binding db-password-staging \
  --member="serviceAccount:migrator@my-project-id.iam.gserviceaccount.com" \
  --role="roles/secretmanager.secretAccessor"

# The migration job itself:
DB_PASSWORD=$(gcloud secrets versions access latest --secret=db-password-staging)
gcloud sql users set-password app-user --instance=staging-instance --password="$DB_PASSWORD"

The password is generated locally with openssl and piped straight into Secret Manager without ever being echoed to a terminal or written to disk, and the only identity granted read access is the specific service account that needs it, consistent with the per-secret access pattern above.

Real-world scenario: encrypting a Terraform state backend's sensitive outputs#

A platform team needs a Terraform module's sensitive outputs (a generated API token) stored somewhere encrypted and access-controlled, without adding it to the state file in plaintext or committing it anywhere:

• Generate the value inside the Terraform run, and immediately store it as a new Secret Manager version rather than leaving it only in Terraform state
• Grant secretAccessor only to the specific consuming service's runtime identity, not to the CI pipeline's broader deploy identity
• Confirm the secret's replication policy matches the consuming service's actual regions (--replication-policy=user-managed with explicit --locations for a service with data-residency requirements, automatic otherwise)
• Set a rotation reminder (--rotation-period + --topics) even though rotation itself is manual, an unrotated long-lived credential is a real, accumulating risk

CI/CD integration recipe: pulling a secret into a build without printing it#

# .github/workflows/build.yml
name: Build with secret
on:
  push:
    branches: [main]

permissions:
  id-token: write
  contents: read

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: google-github-actions/auth@v2
        with:
          workload_identity_provider: projects/<project-number>/locations/global/workloadIdentityPools/github-pool/providers/github-provider
          service_account: ci@my-project-id.iam.gserviceaccount.com
      - id: secrets
        uses: google-github-actions/get-secretmanager-secrets@v2
        with:
          secrets: |-
            NPM_TOKEN:my-project-id/npm-publish-token
      - run: npm publish
        env:
          NODE_AUTH_TOKEN: ${{ steps.secrets.outputs.NPM_TOKEN }}

The dedicated get-secretmanager-secrets action masks the value in GitHub Actions' own log output automatically, a bare gcloud secrets versions access in a run: step doesn't get that masking for free and risks the value appearing in build logs if any step echoes its environment.

Common pitfalls#

• Assigning secretAccessor at the project level instead of per secret. Grants read access to every secret in the project, not just the one intended, see the access-control section above.
• Treating versions destroy as equivalent to versions disable. Destroy is permanent, see the CAUTION above, default to disable unless you're certain.
• Assuming --rotation-period rotates the actual credential value. It only fires a notification; nothing rotates the underlying database password/API key without separate automation subscribed to it.
• Passing a secret value as a literal --data-file string or command-line argument instead of via stdin. Both land in shell history and process listings; --data-file=- with piped input avoids both.
• Choosing the wrong KMS key --purpose at creation and discovering it can't be changed. Confirm symmetric vs. asymmetric, and signing vs. encryption, before creating the key, not after.

Exit codes#

0 success, non-zero on any API/validation error or permission denial, a secrets versions access call against a secret the caller lacks secretAccessor on fails with a PERMISSION_DENIED and non-zero exit, indistinguishable in the exit code alone from the secret simply not existing, check the printed error text for which one actually happened.

When to reach for something else#

For infrastructure-level secrets (a Terraform provider's own credentials, a CI system's bootstrap identity) that need to exist before any application-level secret management is even running, Workload Identity Federation (page 01) removes the need for a stored secret entirely, prefer it over Secret Manager wherever a workload's own identity, rather than a third-party credential it needs to hold, is what's actually being protected. For declarative, reviewable secret and key provisioning (not secret values, which belong in Secret Manager itself, never in Terraform state) across environments, prefer Terraform's google_secret_manager_secret/google_kms_crypto_key resources over a growing shell script of the commands on this page, consistent with the IaC guidance on every earlier page.