A full etcd stops accepting writes, and the whole cluster stops changing.

Every write creates a new revision. Compaction discards old revisions logically; defragmentation then returns the freed space to the disk. They are two separate steps.

1 Revisions pile up

1.8 GB

Data file creeping toward the default 2 GB quota on a busy cluster with no maintenance.

2 Compaction

logical

Tells etcd old revisions are no longer needed. Space becomes reusable inside the file, but the file does not shrink.

3 Defragmentation

physical

Rewrites the file and returns space to the filesystem. Run on one member at a time.

The NOSPACE alarm

Quota exceeded: etcd raises NOSPACE and rejects all writes cluster-wide. After compacting and defragmenting you must also disarm the alarm.

recovery

$ etcdctl alarm list
memberID:8e9e05c52164694d alarm:NOSPACE
$ etcdctl compact $REV     # logical
$ etcdctl defrag --endpoints=$M1   # one member
$ etcdctl alarm disarm

etcd needs housekeeping: compact, then defragment

etcd keeps history: every write produces a new revision so that watches and consistent reads can work. Left alone the database file grows without bound. Compaction tells etcd that revisions older than a given one are no longer needed. That frees space logically inside the file, but the file on disk does not shrink because the underlying storage engine (bbolt) only marks the space reusable.

Defragmentation is the separate operation that rewrites the file and hands space back to the filesystem. Run it on one member at a time, because a member is briefly blocked while it defragments and doing all of them together risks a temporary loss of quorum.

If you skip maintenance, the file eventually reaches the storage quota (2 GB by default, configurable), at which point etcd raises a NOSPACE alarm and rejects all writes. Every kubectl apply, every Pod creation, every scale operation fails until you compact, defragment and explicitly disarm the alarm. Reads continue, which makes this confusing when it first happens.

Real-world note: kubeadm and most distributions configure automatic compaction, and managed services handle it entirely. The risk is mostly on hand-built or heavily customised clusters. Monitor etcd database size and the etcd_mvcc_db_total_size_in_bytes metric and alert well before the quota.