Run several replicas for availability, but let exactly one reconcile.

The controller manager and scheduler use a Lease object as a lock. The holder renews it; if it stops, a standby takes over.

Compare-and-swap

Replicas try to write their identity into the Lease atomically. Only one succeeds. The leader renews before it expires.

A real, bounded gap

Between a leader dying and a standby taking over, nothing reconciles: no new ReplicaSets, no replacements.

who is leading?

$ kubectl -n kube-system get lease \
    kube-controller-manager -o yaml
holderIdentity: cp-2_5e3b...
leaseDurationSeconds: 15

Only one controller manager acts: leader election with a Lease

Production control planes run at least three replicas of the controller manager and scheduler so that one machine failing does not stop reconciliation. But two active copies of a reconciliation loop would issue duplicate and conflicting actions, so only one may be active at a time. Kubernetes solves this with leader election using an ordinary API object, a Lease.

Every replica repeatedly tries to create or update the Lease with its own identity using an atomic compare-and-swap. The one that succeeds is the leader and must renew the lease before it expires. The others sit idle, watching. If the leader crashes or is cut off, it stops renewing, the lease expires after its duration (commonly around fifteen seconds by default), and a standby acquires it.

Notice the cost: this is not instant failover. For the length of the lease expiry nothing is reconciling for that component, so a failed Pod is not replaced and a new Deployment does not get its ReplicaSet until a new leader is elected. Lease duration is therefore a trade-off between fast takeover and avoiding needless leadership flapping during brief network hiccups.

The same mechanism is used by operators you will meet in Deck 7. Gotcha: if every replica runs with leader election disabled by mistake, you get duplicate actions, the classic split-brain of control loops.