External Secrets Operator

What it is and where it fits 🎯#

External Secrets Operator (ESO) is a Kubernetes controller that synchronizes secrets from an external secrets manager — HashiCorp Vault (its own cheat sheet in this series), AWS Secrets Manager, GCP Secret Manager, Azure Key Vault, and dozens more — into native Kubernetes Secret objects, kept continuously up to date. It solves the exact problem this series' Secrets Management & IAM chapter names directly: "every consumer should hold a reference to a secret, never a copy." Without ESO, syncing a Vault value into Kubernetes means someone manually running vault kv get and kubectl create secret — a copy, frozen at that exact moment, with no way for the cluster to know the source value has since rotated. ESO replaces that manual copy with a controller loop: define what to fetch and from where, and it keeps the Kubernetes Secret reconciled to the source automatically, on a refresh interval you control.

This is a genuinely different job from SOPS (its own cheat sheet), which encrypts a secret file so it's safe to commit to a GitOps repository and decrypts it at deploy time — a static, point-in-time value baked into the manifest. ESO instead keeps pulling live from Vault/AWS/GCP/Azure indefinitely, so a credential rotated at the source (a Vault dynamic secret's lease renewal, an AWS Secrets Manager rotation Lambda) reaches the running pod without a human, or even a new deploy, in the loop.

How ESO keeps a Kubernetes Secret in sync#

The pod itself never talks to Vault, AWS, or GCP at all — from its point of view, it's mounting an ordinary Kubernetes Secret, exactly as it always has. All the external-API complexity lives entirely in the controller.

Installation#

# Helm (the officially documented, most common install path)
helm repo add external-secrets https://charts.external-secrets.io
helm repo update
helm install external-secrets external-secrets/external-secrets \
  -n external-secrets --create-namespace

kubectl get pods -n external-secrets                # confirm the controller is Running
kubectl get crds | grep external-secrets.io          # confirm the CRDs installed: SecretStore, ExternalSecret, ...

Core concepts#

A SecretStore pointing at Vault#

# secretstore-vault.yaml
apiVersion: external-secrets.io/v1
kind: SecretStore
metadata:
  name: vault-backend
  namespace: default
spec:
  provider:
    vault:
      server: "http://vault.vault-system:8200"
      path: "secret"
      version: "v2"
      auth:
        kubernetes:
          mountPath: "kubernetes"
          role: "checkout-service"        # the exact Vault role from the Vault cheat sheet's K8s auth example
          serviceAccountRef:
            name: "checkout-service"

kubectl apply -f secretstore-vault.yaml
kubectl describe secretstore vault-backend    # check .status.conditions — Ready: True confirms auth succeeded

Note the auth.kubernetes block — ESO authenticates to Vault using the exact same Kubernetes-native ServiceAccount identity mechanism covered in the Vault cheat sheet, not a static Vault token stored as a Kubernetes Secret (which would just relocate the "secret zero" problem rather than solve it).

An ExternalSecret pulling a value into a native Secret#

# external-secret-db-credentials.yaml
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: checkout-db-credentials
  namespace: default
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: vault-backend
    kind: SecretStore
  target:
    name: checkout-db-credentials     # the native Kubernetes Secret ESO creates/manages
    creationPolicy: Owner              # ESO owns this Secret's lifecycle — deletes it if the ExternalSecret is deleted
  data:
    - secretKey: password              # key inside the resulting Kubernetes Secret
      remoteRef:
        key: checkout/db-credentials   # path inside Vault's KV engine
        property: password             # specific field within that KV entry

kubectl apply -f external-secret-db-credentials.yaml
kubectl get externalsecret checkout-db-credentials       # SecretSynced: True means it's working
kubectl get secret checkout-db-credentials -o yaml         # the resulting native Secret ESO wrote
kubectl describe externalsecret checkout-db-credentials    # events/conditions on failure — check this FIRST when debugging

Sample output — a healthy ExternalSecret#

Representative shape of kubectl get externalsecret output (exact column widths/timestamps vary by version):

READY: False with a recent refresh interval almost always means the previous sync succeeded and is still being served — check kubectl describe for the actual failing condition before assuming an outage.

Pulling a whole path with dataFrom#

spec:
  dataFrom:
    - extract:
        key: checkout/db-credentials     # every key/value pair at this Vault path becomes a Secret key

Use data (the earlier example) when you want to rename or cherry-pick specific fields into the resulting Secret; use dataFrom.extract when the source path's keys should map straight across one-to-one and there's no reason to enumerate them by hand.

Real-world scenario: replacing a manually-copied Secret#

The checkout-service team currently updates a Kubernetes Secret by hand every time the underlying Vault value rotates — a real, common "before" state:

# The BEFORE state — a one-time manual copy, already stale the moment Vault's value changes
kubectl create secret generic checkout-db-credentials --from-literal=password=v1-original-password

Fixing it with ESO:

• Confirm Vault's Kubernetes auth method already has a role scoped to checkout-service's exact ServiceAccount (see the Vault cheat sheet) — ESO reuses this, it doesn't need its own separate identity.
• Apply a SecretStore pointing at that Vault role and mount path.
• Apply an ExternalSecret with a refreshInterval shorter than how often the underlying value actually rotates — a 1h interval against a secret that rotates every 15 minutes still leaves a real staleness window.
• Delete the manually-created Secret and confirm ESO's creationPolicy: Owner version replaced it cleanly.
• Update the Deployment's rollout strategy — a Secret's value changing does not automatically restart pods already using it as an env var (only a volume-mounted Secret updates live); pair this with a checksum annotation or a tool like Reloader if pods must pick up rotations without a manual restart.

Real-world scenario: multi-cloud without changing application code#

A platform team supports teams on both AWS and on-prem Vault, and wants application manifests to look identical regardless of which backend a given cluster uses:

# The ExternalSecret is IDENTICAL across environments — only the SecretStore differs per cluster
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: payments-api-key
spec:
  secretStoreRef:
    name: backend-store    # "backend-store" resolves to Vault on-prem, AWS SM in the cloud cluster
    kind: ClusterSecretStore
  target:
    name: payments-api-key
  data:
    - secretKey: api-key
      remoteRef:
        key: payments/api-key

The application-facing contract (a Secret named payments-api-key with an api-key field) never changes — only the cluster-level ClusterSecretStore definition differs, which platform engineers manage centrally rather than every application team hand-rolling its own backend integration.

CI/CD integration recipe — verifying sync health as a deploy gate#

# .github/workflows/deploy.yml (excerpt)
- name: Wait for ExternalSecret to sync before proceeding
  run: |
    kubectl wait --for=condition=Ready externalsecret/checkout-db-credentials \
      --timeout=60s -n default

Gating a deploy on the ExternalSecret's own Ready condition catches a broken Vault auth binding or a typo'd remote path before the application pod starts crash-looping on a missing/empty secret — a much clearer failure signal than debugging it from inside the application's own logs.

Debugging a failed sync#

kubectl describe externalsecret checkout-db-credentials     # ALWAYS start here — the Events section names the real cause
kubectl logs -n external-secrets deploy/external-secrets     # controller-wide logs if the resource's own events are unclear
kubectl get secretstore vault-backend -o jsonpath='{.status.conditions}'   # confirm the STORE itself is Ready, not just the ExternalSecret

Real-world scenario: least-privilege scoping per namespace#

A platform team supporting several application teams wants each namespace's ExternalSecrets to only be able to reach secrets that specific team actually owns — not a shared, overly broad store:

# Namespaced SecretStore, not ClusterSecretStore — scoped to exactly this namespace
apiVersion: external-secrets.io/v1
kind: SecretStore
metadata:
  name: checkout-team-vault
  namespace: checkout
spec:
  provider:
    vault:
      server: "http://vault.vault-system:8200"
      path: "secret"
      version: "v2"
      auth:
        kubernetes:
          mountPath: "kubernetes"
          role: "checkout-team-role"          # a Vault role scoped ONLY to secret/checkout/* paths
          serviceAccountRef:
            name: "checkout-service"

Pairing a namespaced SecretStore with a Vault role whose policy is scoped to that team's own path prefix means a compromised ExternalSecret in the checkout namespace still can't reach the billing team's secrets — the same least-privilege boundary this series applies repeatedly, enforced at two layers (Kubernetes RBAC on the CRD, and Vault's own policy) rather than relying on either alone.

Common pitfalls#

• Forgetting creationPolicy: Owner semantics. Deleting an ExternalSecret with Owner policy also deletes the Secret it manages — a real risk if something else was also reading that same Secret. Use creationPolicy: Merge if the Secret needs to coexist with keys managed by something else.
• Env-var consumers not picking up rotations. See the warning above — this is the single most common "why didn't my credential rotate" support ticket with ESO in production.
• A SecretStore scoped to the wrong namespace. SecretStore is namespaced; an ExternalSecret in a different namespace can't reference it — use ClusterSecretStore for anything shared across namespaces.
• Not checking kubectl describe externalsecret first. The SecretSynced/Ready condition and its message almost always names the exact failure (auth denied, path not found) far faster than guessing.

Exit codes and when to reach for something else#

ESO is a controller, not a CLI tool with process exit codes to check in a script — health is read from the ExternalSecret/SecretStore resources' own .status.conditions, as shown above. If the goal is encrypting a secret to commit safely to a GitOps repo rather than syncing a live value from a running secrets manager, reach for SOPS instead (its own cheat sheet) — no controller or backend API dependency at runtime. If the cluster genuinely has no external secrets manager at all and never will, ESO has nothing to sync from — that's a signal to stand up Vault (or a cloud-native equivalent) first, not a reason to skip ESO once one exists.