Lose etcd without a backup and the cluster has amnesia.

etcd is a strongly consistent key-value store holding every object. Pods keep running without it for a while, but nothing can be changed, scheduled or healed.

What is stored (key layout)

/registry/pods/shop/checkout-7d9f8-a	Pod object
/registry/deployments/shop/checkout	Deployment
/registry/secrets/shop/payments-db	Secret (encrypt at rest!)

If etcd is lost

No record of what should run, where, or how it is configured.

Managed clusters

GKE, EKS and AKS operate and back up etcd for you.

Self-managed (kubeadm, RKE2)

Snapshots are your job: schedule them, copy off-node, and rehearse a restore.

control-plane node: etcd snapshot

$ ETCDCTL_API=3 etcdctl snapshot save /backup/etcd.db \
    --endpoints=https://127.0.0.1:2379 \
    --cacert=/etc/kubernetes/pki/etcd/ca.crt \
    --cert=/etc/kubernetes/pki/etcd/server.crt \
    --key=/etc/kubernetes/pki/etcd/server.key
Snapshot saved at /backup/etcd.db
$ etcdctl snapshot status /backup/etcd.db -w table
+---------+----------+------------+------------+
|  HASH   | REVISION | TOTAL KEYS | TOTAL SIZE |
+---------+----------+------------+------------+
| 1f4e9a1 |   482913 |       2174 |     18 MB  |
+---------+----------+------------+------------+

An etcd snapshot is only a backup once you have restored from it successfully, so rehearse it before you need it.

etcd is the cluster's entire memory

etcd is a distributed, strongly consistent key-value store and it is the source of truth for the cluster. Every object (Pods, Deployments, Services, Secrets) is a key under /registry. The API server is its only client. It favours consistency over availability: if etcd cannot reach a quorum, writes stop rather than risk two members disagreeing.

That is why losing etcd is catastrophic. Running containers on nodes continue for a while because the kubelet keeps them going, but the cluster cannot create, change, schedule or heal anything, and if the data is truly gone it has no memory of what should exist. On GKE, EKS and AKS the provider runs and backs up etcd, which is a major part of the managed-service value. On kubeadm, RKE2 or k3s clusters, you must schedule snapshots yourself and copy them off the node.

The terminal shows the snapshot command and the verification step. A snapshot file you have never restored is a hope, not a backup: periodically restore to a scratch cluster and check that objects come back. Deck 6 covers Velero for application-level backup, which complements but does not replace etcd snapshots for self-managed control planes.

Security gotcha: Secrets are stored in etcd base64 encoded. Enable encryption at rest with a KMS provider, and treat etcd snapshots as secrets themselves, because they contain every Secret in the cluster.