# External Secrets Operator Cheat Sheet

> **Tool:** External Secrets Operator (ESO)
> **Category:** Security & Compliance
> **Verified against:** Official docs — external-secrets.io/latest/introduction/getting-started,
> .../latest/api/externalsecret, .../latest/api/secretstore — 2026-09-05 (a Kubernetes controller +
> CRDs, not a local CLI — verified against current official docs and CRD schemas, not a `--help` run)
> **Official docs:** https://external-secrets.io/latest/

## What it is and where it fits 🎯

External Secrets Operator (ESO) is a Kubernetes controller that synchronizes secrets from an external
secrets manager — **HashiCorp Vault** (its own cheat sheet in this series), AWS Secrets Manager, GCP
Secret Manager, Azure Key Vault, and dozens more — into native Kubernetes `Secret` objects, kept
continuously up to date. It solves the exact problem this series'
[Secrets Management & IAM](/tutorials/devsecops/04-secrets-management-and-iam) chapter names directly:
"every consumer should hold a *reference* to a secret, never a *copy*." Without ESO, syncing a Vault
value into Kubernetes means someone manually running `vault kv get` and `kubectl create secret` — a copy,
frozen at that exact moment, with no way for the cluster to know the source value has since rotated. ESO
replaces that manual copy with a controller loop: define what to fetch and from where, and it keeps the
Kubernetes `Secret` reconciled to the source automatically, on a refresh interval you control.

This is a genuinely different job from **SOPS** (its own cheat sheet), which encrypts a secret file so
it's safe to commit to a GitOps repository and decrypts it at deploy time — a static, point-in-time
value baked into the manifest. ESO instead keeps pulling live from Vault/AWS/GCP/Azure indefinitely, so a
credential rotated at the source (a Vault dynamic secret's lease renewal, an AWS Secrets Manager rotation
Lambda) reaches the running pod without a human, or even a new deploy, in the loop.

## How ESO keeps a Kubernetes Secret in sync

```mermaid
sequenceDiagram
    participant ES as ExternalSecret (CRD)
    participant ESO as ESO Controller
    participant SS as SecretStore (CRD)
    participant Ext as External backend<br/>(Vault, AWS SM, GCP SM...)
    participant K8s as Kubernetes Secret

    ES->>ESO: Declares WHAT to fetch + refreshInterval
    SS->>ESO: Declares HOW to authenticate + WHERE
    loop Every refreshInterval
        ESO->>Ext: Fetch current secret value
        Ext-->>ESO: Latest value
        ESO->>K8s: Create/update the native Secret
    end
    Note over K8s: Pods mount this Secret exactly<br/>like any other — no ESO awareness<br/>needed inside application code.

    classDef info fill:#e5f0fa,stroke:#1d6fb8,color:#10161c
    classDef ok fill:#e5f5ea,stroke:#1f8a4c,color:#10161c
    class ES,SS info
    class K8s ok
```

The pod itself never talks to Vault, AWS, or GCP at all — from its point of view, it's mounting an
ordinary Kubernetes `Secret`, exactly as it always has. All the external-API complexity lives entirely
in the controller.

## Installation

```bash
# Helm (the officially documented, most common install path)
helm repo add external-secrets https://charts.external-secrets.io
helm repo update
helm install external-secrets external-secrets/external-secrets \
  -n external-secrets --create-namespace

kubectl get pods -n external-secrets                # confirm the controller is Running
kubectl get crds | grep external-secrets.io          # confirm the CRDs installed: SecretStore, ExternalSecret, ...
```

## Core concepts

| Resource | Scope | Purpose |
|---|---|---|
| **SecretStore** | Namespaced | Defines *how* to authenticate to one backend (Vault, AWS SM, ...) and *where* — one per namespace/backend combo |
| **ClusterSecretStore** | Cluster-wide | Same as `SecretStore` but usable by `ExternalSecret`s in any namespace — avoids repeating identical store config per namespace |
| **ExternalSecret** | Namespaced | Declares *what* to fetch from a referenced store, on what `refreshInterval`, and what native `Secret` to write it into |
| **PushSecret** | Namespaced | The reverse direction — pushes a Kubernetes `Secret`'s value *out* to an external backend (less common, used for cluster-generated credentials) |

## A SecretStore pointing at Vault

```yaml
# secretstore-vault.yaml
apiVersion: external-secrets.io/v1
kind: SecretStore
metadata:
  name: vault-backend
  namespace: default
spec:
  provider:
    vault:
      server: "http://vault.vault-system:8200"
      path: "secret"
      version: "v2"
      auth:
        kubernetes:
          mountPath: "kubernetes"
          role: "checkout-service"        # the exact Vault role from the Vault cheat sheet's K8s auth example
          serviceAccountRef:
            name: "checkout-service"
```

```bash
kubectl apply -f secretstore-vault.yaml
kubectl describe secretstore vault-backend    # check .status.conditions — Ready: True confirms auth succeeded
```

Note the `auth.kubernetes` block — ESO authenticates to Vault using the exact same Kubernetes-native
ServiceAccount identity mechanism covered in the Vault cheat sheet, not a static Vault token stored as a
Kubernetes `Secret` (which would just relocate the "secret zero" problem rather than solve it).

## An ExternalSecret pulling a value into a native Secret

```yaml
# external-secret-db-credentials.yaml
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: checkout-db-credentials
  namespace: default
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: vault-backend
    kind: SecretStore
  target:
    name: checkout-db-credentials     # the native Kubernetes Secret ESO creates/manages
    creationPolicy: Owner              # ESO owns this Secret's lifecycle — deletes it if the ExternalSecret is deleted
  data:
    - secretKey: password              # key inside the resulting Kubernetes Secret
      remoteRef:
        key: checkout/db-credentials   # path inside Vault's KV engine
        property: password             # specific field within that KV entry
```

```bash
kubectl apply -f external-secret-db-credentials.yaml
kubectl get externalsecret checkout-db-credentials       # SecretSynced: True means it's working
kubectl get secret checkout-db-credentials -o yaml         # the resulting native Secret ESO wrote
kubectl describe externalsecret checkout-db-credentials    # events/conditions on failure — check this FIRST when debugging
```

## Sample output — a healthy ExternalSecret

Representative shape of `kubectl get externalsecret` output (exact column widths/timestamps vary by
version):

```
NAME                       STORE            REFRESH INTERVAL   STATUS         READY
checkout-db-credentials    vault-backend    1h                 SecretSynced   True
```

`READY: False` with a recent refresh interval almost always means the *previous* sync succeeded and is
still being served — check `kubectl describe` for the actual failing condition before assuming an outage.

## Pulling a whole path with `dataFrom`

```yaml
spec:
  dataFrom:
    - extract:
        key: checkout/db-credentials     # every key/value pair at this Vault path becomes a Secret key
```

Use `data` (the earlier example) when you want to rename or cherry-pick specific fields into the
resulting `Secret`; use `dataFrom.extract` when the source path's keys should map straight across
one-to-one and there's no reason to enumerate them by hand.

## Real-world scenario: replacing a manually-copied Secret

The `checkout-service` team currently updates a Kubernetes `Secret` by hand every time the underlying
Vault value rotates — a real, common "before" state:

```bash
# The BEFORE state — a one-time manual copy, already stale the moment Vault's value changes
kubectl create secret generic checkout-db-credentials --from-literal=password=v1-original-password
```

Fixing it with ESO:

- [ ] Confirm Vault's Kubernetes auth method already has a role scoped to `checkout-service`'s exact
      ServiceAccount (see the Vault cheat sheet) — ESO reuses this, it doesn't need its own separate identity.
- [ ] Apply a `SecretStore` pointing at that Vault role and mount path.
- [ ] Apply an `ExternalSecret` with a `refreshInterval` shorter than how often the underlying value
      actually rotates — a 1h interval against a secret that rotates every 15 minutes still leaves a real
      staleness window.
- [ ] Delete the manually-created `Secret` and confirm ESO's `creationPolicy: Owner` version replaced it cleanly.
- [ ] Update the Deployment's rollout strategy — a `Secret`'s value changing does **not** automatically
      restart pods already using it as an env var (only a volume-mounted `Secret` updates live); pair this
      with a checksum annotation or a tool like Reloader if pods must pick up rotations without a manual restart.

> [!WARNING]
> **A rotated `Secret` doesn't restart pods that consumed it as an environment variable.** Kubernetes only
> re-projects a volume-mounted `Secret`'s new content into already-running pods (eventually, via kubelet's
> sync loop) — an env var was read once, at container start, and never updates. If a workload needs to
> react to a rotation without a full redeploy, mount the secret as a volume and have the app watch the
> file, or run a controller like Reloader/Stakater to trigger a rollout on change.

## Real-world scenario: multi-cloud without changing application code

A platform team supports teams on both AWS and on-prem Vault, and wants application manifests to look
identical regardless of which backend a given cluster uses:

```yaml
# The ExternalSecret is IDENTICAL across environments — only the SecretStore differs per cluster
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: payments-api-key
spec:
  secretStoreRef:
    name: backend-store    # "backend-store" resolves to Vault on-prem, AWS SM in the cloud cluster
    kind: ClusterSecretStore
  target:
    name: payments-api-key
  data:
    - secretKey: api-key
      remoteRef:
        key: payments/api-key
```

The application-facing contract (a `Secret` named `payments-api-key` with an `api-key` field) never
changes — only the cluster-level `ClusterSecretStore` definition differs, which platform engineers manage
centrally rather than every application team hand-rolling its own backend integration.

## CI/CD integration recipe — verifying sync health as a deploy gate

```yaml
# .github/workflows/deploy.yml (excerpt)
- name: Wait for ExternalSecret to sync before proceeding
  run: |
    kubectl wait --for=condition=Ready externalsecret/checkout-db-credentials \
      --timeout=60s -n default
```

Gating a deploy on the `ExternalSecret`'s own `Ready` condition catches a broken Vault auth binding or a
typo'd remote path *before* the application pod starts crash-looping on a missing/empty secret — a much
clearer failure signal than debugging it from inside the application's own logs.

## Debugging a failed sync

```bash
kubectl describe externalsecret checkout-db-credentials     # ALWAYS start here — the Events section names the real cause
kubectl logs -n external-secrets deploy/external-secrets     # controller-wide logs if the resource's own events are unclear
kubectl get secretstore vault-backend -o jsonpath='{.status.conditions}'   # confirm the STORE itself is Ready, not just the ExternalSecret
```

| Symptom | Likely cause |
|---|---|
| `ExternalSecret` stuck `SecretSynced: False`, event mentions "permission denied" | The Vault role's `bound_service_account_names`/namespace doesn't match the ServiceAccount actually referenced |
| `SecretStore` itself shows `Ready: False` | Backend unreachable, or the auth method/role doesn't exist yet in Vault |
| `ExternalSecret` is `Ready` but the `Secret`'s data looks stale | `refreshInterval` is longer than expected — check the actual value, not an assumption |
| A referenced `remoteRef.property` errors as "not found" | The field name inside the Vault KV entry doesn't match — `vault kv get` the path directly to confirm the real field names |

> [!TIP]
> **Check the `SecretStore`'s own `Ready` condition before debugging an individual `ExternalSecret`.**
> A broken backend connection affects every `ExternalSecret` referencing that store at once — fixing the
> store's auth binding once is faster than troubleshooting several seemingly-unrelated `ExternalSecret`
> failures that all trace back to the same root cause.

## Real-world scenario: least-privilege scoping per namespace

A platform team supporting several application teams wants each namespace's `ExternalSecret`s to only be
able to reach secrets that specific team actually owns — not a shared, overly broad store:

```yaml
# Namespaced SecretStore, not ClusterSecretStore — scoped to exactly this namespace
apiVersion: external-secrets.io/v1
kind: SecretStore
metadata:
  name: checkout-team-vault
  namespace: checkout
spec:
  provider:
    vault:
      server: "http://vault.vault-system:8200"
      path: "secret"
      version: "v2"
      auth:
        kubernetes:
          mountPath: "kubernetes"
          role: "checkout-team-role"          # a Vault role scoped ONLY to secret/checkout/* paths
          serviceAccountRef:
            name: "checkout-service"
```

Pairing a namespaced `SecretStore` with a Vault role whose policy is scoped to that team's own path prefix
means a compromised `ExternalSecret` in the `checkout` namespace still can't reach the `billing` team's
secrets — the same least-privilege boundary this series applies repeatedly, enforced at two layers
(Kubernetes RBAC on the CRD, and Vault's own policy) rather than relying on either alone.

## Common pitfalls

- **Forgetting `creationPolicy: Owner` semantics.** Deleting an `ExternalSecret` with `Owner` policy also
  deletes the `Secret` it manages — a real risk if something else was also reading that same `Secret`.
  Use `creationPolicy: Merge` if the `Secret` needs to coexist with keys managed by something else.
- **Env-var consumers not picking up rotations.** See the warning above — this is the single most common
  "why didn't my credential rotate" support ticket with ESO in production.
- **A `SecretStore` scoped to the wrong namespace.** `SecretStore` is namespaced; an `ExternalSecret` in a
  different namespace can't reference it — use `ClusterSecretStore` for anything shared across namespaces.
- **Not checking `kubectl describe externalsecret` first.** The `SecretSynced`/`Ready` condition and its
  message almost always names the exact failure (auth denied, path not found) far faster than guessing.

## Exit codes and when to reach for something else

ESO is a controller, not a CLI tool with process exit codes to check in a script — health is read from
the `ExternalSecret`/`SecretStore` resources' own `.status.conditions`, as shown above. If the goal is
encrypting a secret to commit safely to a GitOps repo rather than syncing a live value from a running
secrets manager, reach for **SOPS** instead (its own cheat sheet) — no controller or backend API
dependency at runtime. If the cluster genuinely has no external secrets manager at all and never will,
ESO has nothing to sync from — that's a signal to stand up **Vault** (or a cloud-native equivalent)
first, not a reason to skip ESO once one exists.
