Computer Architecture & the CPU

Table of Contents#

1. Why This Series Starts Below the Operating System
2. Anatomy of a CPU — Cores, Threads, and Clock Speed
3. Instruction Set Architectures — Why ARM Is Suddenly Everywhere in the Cloud
4. Simultaneous Multithreading — One Core Pretending to Be Two
5. The Instruction Cycle — Pipelining, Out-of-Order Execution, and Branch Prediction
6. The Cache Hierarchy — Why Memory Locality Decides Performance
7. Interrupts — How Hardware Gets the CPU's Attention
8. Context Switching — The Real Cost of Multitasking
9. NUMA — When "One Big Pool of Memory" Is a Convenient Lie
10. Multi-Core Scaling and Amdahl's Law
11. Frequency Scaling, Turbo Boost, and Credit-Based Throttling
12. Idle Power States — Why "Idle" Isn't Free, and Isn't Instant
13. CPU Affinity and Pinning — Taking Manual Control
14. Reading CPU Topology on a Real Machine
15. CPU-Bound vs. I/O-Bound Workloads — Classifying What You're Actually Running
16. How These Concepts Show Up on the Cloud Bill
17. Common Mistakes and Interview Traps
18. Worked Practice Problems
19. Summary and What's Next

Why This Series Starts Below the Operating System#

Every other tutorial series on this site starts from something the operating system already provides — a process, a socket, a filesystem, a scheduled pod. This series is the one layer underneath all of them: the actual physical hardware a Linux kernel is managing, and the operating-system machinery that turns a pile of silicon into something that can run a fleet of services at all. An SRE or platform engineer who has never had to reason about this layer can still operate successfully for a long time — until the day a service's latency graph shows a mysterious sawtooth pattern that has nothing to do with the application code, a kubectl top node reading doesn't match what the underlying host is actually doing, or a cloud vendor's support engineer asks "have you checked CPU steal time?" and the honest answer is "I don't know what that is." This chapter, and this series, exists to close that gap.

The running example across this chapter — and, where relevant, the rest of this series — is the same fictional e-commerce platform used throughout this site's Terraform, Kubernetes, Observability, Networking, and Incident Management series: three services, checkout-service, catalog-service, and inventory-service, running on a fleet of cloud virtual machines. Every one of those VMs is, underneath the guest operating system, a slice of a real physical server sitting in a real data center, with real CPU cores, real memory chips, and real disks — and every performance problem this platform will ever have eventually traces back to how those physical resources are being shared, scheduled, and contended for. This chapter starts at the most fundamental resource of all: the CPU itself.

Where this series goes from here: this chapter covers the CPU in isolation. Part 2 builds directly on it, covering how the Linux scheduler actually decides which thread runs on which core. Part 3 does the same for memory — physical and virtual memory, paging, and the page cache. Part 4 moves to storage and filesystems, Part 5 to virtualization (how a hypervisor slices real hardware into the VMs this chapter has been implicitly describing all along), Part 6 to the boot process and kernel architecture that ties every earlier piece together into one running system, and Part 7 closes the series by applying all six prior chapters to real production incidents — CPU steal time, OOM kills, disk saturation — the way a platform engineer actually encounters them, mixed together, during an incident.

Anatomy of a CPU — Cores, Threads, and Clock Speed#

A modern server CPU is not one thing doing one thing at a time — it's a package containing multiple independent cores, each one a genuinely separate processing unit capable of executing its own stream of instructions in parallel with every other core on the chip. A cloud instance advertised as having "8 vCPUs" is, in the simplest case, a slice of a physical host's cores handed to that one virtual machine (Part 5 of this series covers exactly how that slicing works under a hypervisor).

What to notice: every core has its own private, small, extremely fast cache, but they all ultimately share one path down to main memory — this shared path is exactly why "how many cores does it have" is an incomplete question, and Section 6 below covers why.

Three numbers show up on every spec sheet, and practitioners routinely conflate them:

From the Trenches: a platform team migrated checkout-service from an older cloud instance family to a newer one advertised as having the same vCPU count at a lower hourly price, expecting an easy cost win. Median request latency got measurably worse. The immediate cause was that the new instance family packed vCPUs more densely per physical core (more simultaneous-multithreading sharing per real core), so the same nominal vCPU count delivered less real execution throughput under checkout-service's CPU-heavy request-validation logic. The underlying, two-levels-deep condition was that nobody on the team had ever looked past "vCPU count" as the unit of comparison between instance families — the cloud provider's own documentation listed the physical-core-to-vCPU ratio per family, but it had never been part of the team's sizing checklist because the number had always looked comparable before. The fix wasn't reverting the migration; it was adding "physical core ratio, not just vCPU count" to the instance-family comparison the team now runs before any resize.

Instruction Set Architectures — Why ARM Is Suddenly Everywhere in the Cloud#

Every CPU implements an instruction set architecture (ISA) — the fixed vocabulary of operations (add, load, store, branch) it understands at the hardware level. For most of the server-computing era, the practical answer for anyone running production workloads was simply x86-64 (Intel and AMD's shared, backward-compatible architecture), and the ISA question rarely came up in day-to-day platform work. That changed with the rise of cloud-vendor-built ARM server chips — AWS Graviton, Google Axion, Azure Cobalt — which are now a routine, cost-driven instance choice rather than a niche experiment.

The two architecture families take genuinely different design philosophies:

Why this genuinely matters for a platform/DevOps engineer, not just a hardware curiosity: container images are compiled for one specific ISA. A container image built on an x86-64 laptop and pushed straight to a Graviton (ARM) node will not run at all — this is not a performance question, it's a hard compatibility failure (exec format error). Modern container tooling (docker buildx, multi-architecture image manifests) solves this by building and publishing one image tag that actually contains multiple architecture-specific variants, letting the container runtime pull whichever one matches the node it's scheduling onto — but that only works if the build pipeline was actually configured to produce both variants in the first place.

From the Trenches: a team migrated half of catalog-service's Kubernetes node pool to Graviton instances to cut compute cost, following the cloud provider's published price-performance numbers, and the rollout paged on-call within an hour — a fraction of pods stuck in CrashLoopBackOff with an exec format error in the container logs. The immediate cause was that catalog-service's image was still being built as a single-architecture amd64 image; the CI pipeline had never been updated to produce an arm64 variant, so any pod the scheduler happened to place on the new Graviton nodes failed immediately. The underlying, two-levels-deep condition was that the cost-optimization initiative had been scoped and approved purely at the infrastructure/node-pool level — nobody had connected it back to the application build pipeline as a dependency, because from the infrastructure team's perspective, "add some cheaper nodes to the pool" looked like a self-contained, low-risk infrastructure change. The fix, beyond the immediate CI update to docker buildx multi-arch builds, was adding "does every deployed image ship a matching multi-arch variant?" as an explicit precondition on the team's node-pool architecture-diversity runbook, not an afterthought discovered by an incident.

A decision framework, not just an incident to avoid:

Simultaneous Multithreading — One Core Pretending to Be Two#

A single CPU core executing one instruction at a time would leave large amounts of its own internal circuitry idle at any given moment — a core has many independent execution units (integer math, floating-point math, memory load/store) and a real program rarely keeps all of them simultaneously busy. Simultaneous multithreading (SMT — Intel's implementation is branded Hyper-Threading) exploits this by letting one physical core present itself to the operating system as two logical processors, each with its own register state, sharing the same underlying execution units.

The genuinely important operational consequence: two logical threads on the same physical core are not two independent cores. When the operating system's scheduler (Part 2 of this series) sees "16 vCPUs" on an SMT-enabled 8-core chip, it is seeing 8 real cores and 16 logical threads competing in pairs for the same underlying execution hardware. Two CPU-bound threads scheduled onto the same physical core (one on each logical thread) will run meaningfully slower, combined, than the same two threads scheduled onto two genuinely separate physical cores — because they're contending for one shared set of execution units, not running in true parallel.

This is precisely why security-sensitive and consistently-latency-sensitive workloads sometimes disable SMT entirely, and why some cloud "compute-optimized" instance families do the same by default — trading raw advertised vCPU count for more predictable, contention-free per-vCPU performance. It also underlies a well-known class of side-channel security vulnerabilities (the Spectre family of attacks, and related research exploiting shared execution-unit and cache state between co-scheduled SMT threads), which is a large part of why some regulated, multi-tenant workloads mandate SMT-disabled instances specifically.

From the Trenches: an SRE team benchmarking inventory-service on a new bare-metal database host saw query throughput roughly 1.3x higher than the old host's per-vCPU count would predict, and assumed the new hardware generation was simply faster silicon. The immediate cause was correct as far as it went — newer silicon is faster — but the bigger factor, discovered only when a teammate cross-checked lscpu's "Thread(s) per core" field, was that the old host had SMT enabled and the new host's database-tier machines were provisioned from an SMT-disabled pool by an infrastructure standard nobody on the database team had been told about. The underlying, two-levels-deep condition was an undocumented provisioning convention that silently changed the effective core-to-vCPU ratio, and no runbook or dashboard surfaced that fact anywhere a database engineer would naturally look. The fix was adding lscpu's thread-per-core output to the standard host-onboarding checklist, so capacity comparisons across host generations account for it explicitly instead of being attributed to vague "newer hardware" intuition.

Interview-ready line: "Simultaneous multithreading lets one physical core present itself as two logical CPUs by sharing its execution units between two independent register states. It genuinely helps mixed workloads where one thread is often waiting, but two truly CPU-bound threads on the same physical core will contend for the same shared hardware — so 16 logical CPUs on an 8-core SMT chip is not the same compute capacity as 16 genuinely independent cores."

The Instruction Cycle — Pipelining, Out-of-Order Execution, and Branch Prediction#

Every single line of application code — a Python function, a Go goroutine, a database query — eventually compiles or interprets down to a sequence of extremely simple CPU instructions (add these two numbers, load this value from this memory address, jump to this instruction if that comparison was true). A CPU core executes those instructions through a repeating cycle, classically described in four stages:

What to notice: every single stage of this cycle depends on memory access — either reading the instruction itself, or reading/writing the data it operates on — which is exactly why the next section (the cache hierarchy) is not a side detail. A CPU core that can theoretically execute billions of instructions per second is entirely bottlenecked by how fast it can actually get instructions and data into its execution units.

Modern CPUs don't run this cycle in the strictly sequential way the diagram above implies for a single instruction — real hardware uses three techniques together to keep execution units busy far more of the time than a naive one-instruction-at-a-time model would allow:

• Pipelining — starting the fetch of the next instruction before the current one has finished executing, like an assembly line where several instructions are simultaneously at different stages of the cycle. A pipeline with, say, 14 stages can have up to 14 different instructions in flight at once, each at a different point in fetch/decode/execute/write-back.
• Out-of-order execution — reordering independent instructions internally to keep execution units busy while a slower instruction (typically one waiting on a cache miss) is still in flight, then reassembling the results in the original program order so the program's observable behavior is unaffected.
• Branch prediction — guessing which way an if statement will resolve before the comparison itself has actually completed, so the pipeline can keep speculatively fetching and executing instructions down the predicted path instead of stalling until the real answer is known. Modern branch predictors are remarkably accurate — often exceeding 95% on typical code — because most real-world branches have a strong statistical bias (a null-check that's almost always false, a retry loop that almost always succeeds on the first try).

A branch misprediction forces the CPU to discard every piece of speculative work it did down the wrong path and restart from the correct instruction — a real, measurable performance cost that scales with how deep the pipeline is (deeper pipelines have more speculative work to throw away on a misprediction). This is a large part of why database query planners, JIT compilers, and performance-sensitive library code go to real effort to produce branch-predictable code paths — sorting data before a filtering loop, for instance, can measurably speed up the loop itself for no algorithmic reason at all, purely because a sorted sequence produces far more predictable branch outcomes than a random one.

From the Trenches: an engineer optimizing checkout-service's discount-eligibility check replaced a series of unpredictable, data-dependent if branches (checking a customer's cart against dozens of promotional rules in an order that varied per request) with a single lookup-table-driven approach, expecting the win to come from reduced instruction count. perf stat's branch-misprediction counter, checked out of curiosity rather than as part of the original hypothesis, showed the real story: the original code's misprediction rate was over 20% (each rule check's outcome depended on unpredictable cart contents), and the rewrite's was under 2%. The measured latency improvement was roughly double what the reduced instruction count alone would predict — the rest came entirely from eliminating pipeline-flushing mispredictions the team hadn't set out to fix and hadn't even known to look for.

Interview-ready line: "Modern CPUs use pipelining, out-of-order execution, and branch prediction together to keep execution units busy far more of the time than a naive one-instruction-at-a-time model would allow. Branch prediction specifically guesses which way a conditional will resolve and speculatively executes down that path — a correct guess is free performance, but a misprediction forces the pipeline to discard all that speculative work and restart, which is why unpredictable, data-dependent branching in a hot path can cost measurably more than its raw instruction count suggests."

The Cache Hierarchy — Why Memory Locality Decides Performance#

Main memory (RAM) is dramatically slower than the CPU core trying to use it — accessing RAM directly costs roughly 200 CPU cycles, during which a 3+ GHz core could otherwise have executed hundreds of instructions. If every single memory access actually had to wait for RAM, modern CPUs would spend the overwhelming majority of their time doing nothing but waiting. The fix is a cache hierarchy: several progressively larger, progressively slower layers of memory sitting between the CPU core and RAM, each one holding a copy of the data most likely to be needed again soon.

What to notice: the jump from L3 cache to RAM is by far the largest latency cliff in the entire hierarchy — roughly 4-5x slower than even the slowest cache layer. This single fact explains an enormous amount of real-world performance-tuning advice: keeping a working data set small enough to fit in cache, processing data sequentially rather than jumping randomly across memory, and reusing a value shortly after first touching it (temporal locality) versus accessing values that are physically close together in memory (spatial locality) are both, at their core, strategies for maximizing how often a memory access is satisfied by a fast cache hit instead of a slow RAM round-trip.

A cache miss — needing data that isn't present in a given cache layer — forces the CPU to check the next, slower layer down, all the way to RAM in the worst case. A workload's cache hit rate is one of the single biggest hidden determinants of real-world throughput for CPU-intensive services, and it's almost entirely invisible from a typical application-level dashboard: two services doing "the same amount of work" by instruction count can perform wildly differently if one has a data-access pattern that stays cache-friendly and the other jumps around memory unpredictably.

A concrete illustration from this platform's own codebase: catalog-service's product-search endpoint originally iterated a large in-memory array of product records scattered across many separately-allocated objects (poor spatial locality — each object access likely means a fresh cache miss), then was rewritten to use a tightly packed, contiguous array of the same data (excellent spatial locality — sequential objects tend to land in the same cache line). No algorithmic change was made at all; the instruction count per request was essentially identical. Median latency dropped by roughly a third purely from the improved cache behavior — a result that would be completely invisible to anyone reasoning only about Big-O algorithmic complexity.

A related, less obvious cache problem is false sharing: cache is managed in fixed-size blocks called cache lines (typically 64 bytes on modern x86 and ARM server chips), and if two threads running on different cores each frequently write to two entirely unrelated variables that happen to live on the same cache line, the cache-coherency hardware has to keep invalidating and re-fetching that line back and forth between the two cores' private caches — even though the threads never actually touch each other's data. It looks, from the outside, exactly like unexplained contention on data that has no logical relationship at all, which makes it a notoriously confusing class of bug to diagnose without knowing this mechanism exists.

Working in the opposite direction from false sharing, modern CPUs also include hardware prefetchers — dedicated circuitry that watches a core's memory-access pattern and speculatively loads data into cache before the program actually asks for it, betting that a sequential or otherwise predictable access pattern will keep continuing. This is a large part of why sequential array iteration measurably outperforms an equivalent linked-list traversal doing the same logical work: the prefetcher can recognize and get ahead of a sequential pattern, but has no way to predict where a linked list's next node lives in memory until the current node's pointer is actually read.

# perf stat surfaces cache behavior directly, not just wall-clock time
perf stat -e cache-references,cache-misses,instructions,cycles \
  -- ./catalog-service-search-benchmark

# A representative (illustrative) result:
#  1,204,830,112  cache-references
#     98,442,201  cache-misses     #  8.17% of all cache references
#  4,821,003,552  instructions
#  2,110,447,908  cycles

Interrupts — How Hardware Gets the CPU's Attention#

A CPU core is always executing something — but most of the events a running system needs to react to (a network packet arriving, a disk finishing a read, a timer firing) happen asynchronously, with no relationship to whatever instruction the CPU happens to be on at that moment. Interrupts are the mechanism hardware uses to say "stop what you're doing right now and come handle this" — a signal, delivered directly to the CPU, that immediately suspends the current instruction stream and jumps to a specific handler routine.

Linux deliberately splits interrupt handling into two halves, for a specific, important reason:

• Top half (the hardware interrupt handler itself) runs with interrupts disabled or heavily restricted, so it is kept deliberately, aggressively minimal — acknowledge the device, copy the bare minimum of data out, and get out as fast as possible. Spending too long here blocks other interrupts (including timer interrupts the scheduler depends on) from being serviced at all.
• Bottom half (softirqs, run by the ksoftirqd kernel thread when load is high) does the actual heavier processing — walking a packet through the TCP/IP stack, for instance — outside of the restrictive interrupt context, where it can be preempted and doesn't block other hardware interrupts.

This split is directly observable and directly operationally relevant: top's %si (softirq) CPU-time column and /proc/interrupts' per-device interrupt counts are two of the most under-used diagnostic signals in a typical SRE's toolkit, and both come straight from this mechanism.

From the Trenches: during a traffic spike, checkout-service's host showed CPU utilization near 100% in monitoring, but application-level profiling showed the service's own request-handling code was nowhere near saturated — most of the "busy" CPU time, on closer inspection with top, was sitting in the %si (softirq) column, not %us (user) or %sy (system) time attributable to the application. The immediate cause was that a single network interface's interrupts were pinned to one specific CPU core by default (no interrupt spreading configured), so all inbound packet processing for the entire host funneled through that one core's softirq handling, saturating it long before the other cores handling application logic were anywhere near their own limits. The underlying, two-levels-deep condition was that the host's default IRQ affinity had never been tuned for a network-heavy workload — it was a generic default appropriate for a lightly-loaded host, silently wrong for a host running a high-throughput public-facing service, and nothing in the standard CPU-utilization dashboard distinguished "one core softirq-saturated" from "genuinely out of CPU capacity" without drilling into the per-core, per-category breakdown. The fix was enabling irqbalance (which spreads interrupt handling across available cores) and, for the highest-throughput hosts, manually pinning specific interrupt queues to specific cores reserved for that purpose (a technique sometimes called RSS — Receive Side Scaling — tuning).

# Confirm interrupt distribution across cores directly
cat /proc/interrupts | grep eth0
#  eth0-TxRx-0   45231   12   8    9   ...   <-- heavily skewed toward CPU 0

# Check whether RSS (Receive Side Scaling) is spreading queues across
# multiple cores in the first place — a NIC with only one queue can't
# be spread no matter how irqbalance or manual affinity is configured
ethtool -l eth0
# Channel parameters for eth0:
# Combined:    1        <-- only one RX/TX queue; a real capacity limit

Context Switching — The Real Cost of Multitasking#

A single CPU core physically executes exactly one instruction stream at a time — yet a typical Linux host runs dozens or hundreds of processes and threads that all appear to be running "simultaneously." The mechanism that creates this illusion is the context switch: the operating system's scheduler (the full subject of Part 2) periodically stops the currently running thread, saves its complete execution state, and loads a different thread's saved state so it can resume exactly where it left off.

A context switch is not free — it costs real, measurable time, for reasons that compound:

• Register save/restore — every CPU register, the program counter, and the stack pointer for the outgoing thread must be written to memory, and the incoming thread's saved values loaded back in.
• Pipeline flush — the speculative work the CPU's pipeline had in flight for the outgoing thread (Section 5's out-of-order execution) is discarded; the incoming thread starts with a cold pipeline.
• Cache and TLB pollution — the outgoing thread's data may have been comfortably resident in L1/L2 cache and the CPU's TLB (translation lookaside buffer — a small cache of recent virtual-to-physical memory address translations, covered fully in Part 3); the incoming thread's own working set now has to reload into those same limited-size caches, evicting the outgoing thread's data. This "cold cache" penalty on the next several memory accesses is frequently larger than the context switch's own direct register-save cost.

Context switches themselves cost low single-digit microseconds of pure overhead, but the cache/TLB pollution effect can add a meaningfully larger indirect cost on top, which is exactly why an excessively context-switch-heavy workload measurably underperforms one that does the identical amount of work with fewer, larger chunks of uninterrupted CPU time.

From the Trenches: a team running inventory-service with an aggressively high thread-pool size (several hundred worker threads on an 8-core host, reasoning "more threads means more concurrency") saw throughput actually decrease under peak load compared to a much smaller thread pool, which was counterintuitive to everyone involved. The immediate cause, visible in vmstat's context-switch counter, was that with far more runnable threads than physical cores, the scheduler was forced into constant, rapid context switching just to give every thread a fair slice of time — the vast majority of each core's time was going to switching overhead and cold-cache reloading rather than actual request-handling work. The underlying, two-levels-deep condition was a mental model borrowed from I/O-bound workloads (where more threads genuinely does help, because most threads are blocked waiting rather than competing for CPU) applied incorrectly to inventory-service's actual profile, which was substantially CPU-bound — its threads were rarely blocked, so adding more of them past the physical core count added pure scheduling overhead with no corresponding concurrency benefit. The fix was sizing the thread pool close to the physical core count for the CPU-bound portions of the workload, with a separate, larger pool reserved specifically for the genuinely I/O-bound calls.

vmstat 1
# procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
#  r  b   swpd   free   buff  cache   si   so    bi    bo   in    cs us sy id wa st
#  24 0      0  81234  12044 512300    0    0     0     4  9821 48213 71  9 18  2  0
#                                                              ^^^^^
#                                          context switches/sec — climbing here with
#                                          no matching throughput gain is the signal
#                                          from the From the Trenches example above

NUMA — When "One Big Pool of Memory" Is a Convenient Lie#

On any server with more than one physical CPU socket (common on larger database hosts, and on the physical hosts underlying larger cloud instance types), memory is not actually one uniform pool equally reachable from every core — it's physically divided, with each socket having its own directly attached memory. This architecture is called NUMA — Non-Uniform Memory Access.

What to notice: a core on Socket 0 can read memory attached to Socket 1 — the system is still one coherent address space — but it has to cross an inter-socket interconnect to do it, at a meaningful latency penalty (roughly 1.5-2x local-memory latency, according to real hardware measurements) compared to reading its own socket's directly attached memory. A process whose memory happens to be allocated on a remote NUMA node relative to the core it's actually running on pays this penalty on every single memory access, for the entire time it runs there — a quiet, systemic tax that never shows up as an error, only as unexplained latency variance.

Linux's default scheduling and memory-allocation behavior tries to keep a process's memory local to the NUMA node it's running on, but this can break down in specific, real situations: a process migrated between cores on different sockets after its memory was already allocated, a container resource limit that doesn't account for NUMA topology at all, or a workload whose memory footprint is simply larger than one node's local memory and has to spill onto a remote node.

From the Trenches: a database team migrated their primary inventory-service datastore to a larger dual-socket host expecting proportionally better performance from the doubled core and memory count, and instead saw p99 query latency become noticeably less consistent — same median, much wider tail — than on the smaller single-socket host it replaced. The immediate cause, found via numastat, was a high numa_miss rate: the database's large in-memory buffer pool had been allocated without any NUMA awareness, spread across both nodes, so roughly half of all buffer-pool reads from any given core were paying the cross-socket latency penalty. The underlying, two-levels-deep condition was that the single-socket host the team had used for years had made NUMA entirely irrelevant by construction — there was only one node, so the concept had simply never come up in their capacity-planning process, and nobody had flagged "this new host has two sockets" as a fact that changed anything. The fix was configuring the database to use NUMA-aware memory allocation (binding its buffer pool allocation to match the NUMA node its worker threads ran on) rather than relying on default, topology-unaware allocation.

Interview-ready line: "NUMA means memory access time depends on which CPU is asking — a core reading its own socket's directly attached memory is fast, but the same core reading memory attached to a different socket has to cross an inter-socket interconnect, at roughly 1.5-2x the latency. It matters practically because a process migrated or allocated across NUMA nodes pays that penalty on every memory access, invisibly, with no error and no obvious dashboard signal — numastat and numactl --hardware are the tools that make it visible."

Multi-Core Scaling and Amdahl's Law#

Every section so far has assumed that adding more cores, or more threads, is a straightforward way to get more throughput. It isn't — not because parallelism doesn't work, but because almost no real workload is entirely parallelizable, and the portion that genuinely cannot run in parallel puts a hard mathematical ceiling on how much any amount of additional hardware can help. This is Amdahl's Law, formalized by computer architect Gene Amdahl in 1967 and still the single most important reason "just add more cores" is not a universal fix.

The intuition: split a task into a fraction that can be parallelized across many cores, and a fraction that fundamentally cannot (a step that must run sequentially — initializing shared state, merging parallel results back together, waiting on a single lock). No matter how many cores are thrown at the parallelizable fraction, the sequential fraction still takes exactly as long as it always did, and increasingly dominates the total runtime as the parallel portion shrinks toward zero.

What to notice: the "50% parallel" line plateaus hard, approaching a maximum speedup of only 2x no matter how many cores are added — the serial half of the work simply never gets faster. The "99% parallel" line keeps climbing much further before flattening, because its serial fraction is tiny. The practical takeaway for capacity planning: the ceiling on how much a workload benefits from more cores is set by its least parallelizable step, not its average behavior — and identifying that step is usually far more valuable than simply adding hardware and hoping.

From the Trenches: a team scaled inventory-service's stock-reservation endpoint from 4 to 16 cores, expecting close to a 4x throughput improvement based on a synthetic single-function microbenchmark that had shown near-linear scaling. Real production throughput improved by barely 1.4x. The immediate cause, found by profiling the real endpoint rather than the isolated microbenchmark, was that every reservation request ultimately serialized through a single row-level lock on the specific product's inventory count in the database — a step that was entirely absent from the microbenchmark, which had tested the parallelizable validation logic in isolation without ever touching the real, shared, serializing resource. The underlying, two-levels-deep condition was that the microbenchmark had been written to measure "the code," not "the endpoint" — it accurately reflected the parallel fraction's own scaling behavior, but the team had implicitly assumed that fraction represented the whole request, when the real, production-representative serial fraction (the database lock) was the part actually setting the ceiling. The fix wasn't more cores at all — it was reducing lock contention itself (narrowing the lock's scope from the whole product row to just the quantity field, and batching reservation updates), which moved the entire Amdahl's Law ceiling higher rather than chasing a hardware answer to an architecture problem.

Interview-ready line: "Amdahl's Law says the maximum possible speedup from parallelizing a task is capped by the fraction that can't be parallelized — a workload that's 50% serial can never go faster than 2x, no matter how many cores you throw at it, because the serial half's duration never shrinks. Practically, it means the right response to a scaling problem is usually finding and shrinking the serial bottleneck itself, not just adding hardware against it."

Frequency Scaling, Turbo Boost, and Credit-Based Throttling#

A CPU's advertised clock speed is not a single fixed number a core always runs at — modern chips dynamically adjust their actual operating frequency, for two related but distinct reasons that matter differently to a platform engineer.

Dynamic frequency scaling and Turbo Boost (Intel)/Precision Boost (AMD) let a core temporarily run above its base clock speed when there's thermal and power headroom — typically when only a few of the chip's cores are active, so the freed-up thermal budget can be spent pushing the busy cores faster. The corollary matters just as much: under sustained, all-core load, a chip often cannot maintain its single-core turbo frequency across every core simultaneously, and settles to a lower sustained "all-core turbo" frequency instead. A benchmark run briefly, on a mostly-idle chip, can look meaningfully faster than the same workload running for an extended period alongside genuinely full utilization on every other core — a discrepancy that has confused more than one team comparing a short synthetic load test against real sustained production traffic.

Cloud-specific credit-based throttling is a related but entirely separate mechanism, unique to "burstable" cloud instance families (AWS's T-series is the best-known example, with equivalents on other providers). These instance types are deliberately sold at a lower baseline CPU allocation than their vCPU count would suggest, and accumulate CPU credits whenever actual usage stays below that baseline — one credit is roughly equivalent to one vCPU running at 100% for one minute. When usage exceeds the baseline, the instance spends accumulated credits to "burst" above it; once the credit balance is exhausted, the instance is hard-throttled back down to its baseline performance level, regardless of demand, until credits accumulate again.

From the Trenches: a team ran catalog-service's image-processing batch job on a T-family burstable instance because short load tests during development consistently showed excellent performance, well above what the instance's vCPU count and price would normally suggest. In production, the same job — now running continuously for hours instead of in short test bursts — became dramatically slower partway through every run, at a strikingly consistent point. The immediate cause, visible in the cloud provider's CPU-credit-balance metric, was straightforward: the job's sustained CPU usage was well above the instance family's baseline, so it burned through its accumulated credit balance at a predictable rate and hit hard baseline throttling at almost exactly the same elapsed time on every run. The underlying, two-levels-deep condition was that the instance type had been chosen based on a load-testing methodology (short bursts, always followed by idle recovery time that let credits rebuild) that structurally could never have revealed this failure mode — the type of instance, not any configuration on it, was fundamentally mismatched to a sustained-load workload, and nothing about a short test would ever have surfaced that mismatch. The fix was moving the batch job to a fixed-performance (non-burstable) instance family sized for its actual sustained CPU requirement, reserving burstable instances specifically for the bursty, mostly-idle workloads they're economically designed for.

# Inspect current frequency-scaling governor and live frequency range
cpupower frequency-info
# analyzing CPU 0:
#   driver: intel_pstate
#   current policy: frequency should be within 800 MHz and 3.80 GHz
#   the governor "powersave" may decide which speed to use
#   current CPU frequency: 3.41 GHz (asserted by call to hardware)

The governor reported here is the kernel's own policy for how aggressively to scale frequency — performance keeps the core near its maximum available frequency at all times (trading power efficiency for consistently low latency), while powersave (the modern default on most distributions, despite the name, works with hardware-level boost technology rather than against it) lets the hardware itself decide when to boost and when to conserve. Confirming which governor is active is a legitimate first check before concluding a "slow" CPU-bound workload is a hardware capacity problem rather than a power-policy setting.

Idle Power States — Why "Idle" Isn't Free, and Isn't Instant#

Section 11 covered how a CPU scales its frequency up under load. The complementary mechanism handles the opposite case: what a core does when it has genuinely nothing to execute. Rather than sitting fully powered and simply not doing anything, a modern CPU core drops into one of several C-states — progressively deeper sleep states, each saving more power at the cost of taking longer to wake back up when new work arrives.

Why this matters beyond a power bill: the deeper the C-state a core drops into, the longer it takes to wake back up and resume executing — a real, measurable latency cost paid the moment new work (an incoming request, a scheduled thread) actually arrives. For most general-purpose workloads this is invisible, because the wake latency is a tiny fraction of a millisecond and the power savings are a genuine, worthwhile default. For latency-sensitive workloads with bursty, intermittent traffic — exactly checkout-service's profile during a flash sale, with long idle gaps between sudden spikes — that wake-up latency can show up as real, measurable tail-latency jitter on the very first request after an idle period.

A distinct, virtualization-specific wrinkle worth naming now (Part 5 covers the full mechanics): inside a virtual machine, the guest operating system does not control real C-states or P-states at all — those decisions are made by the physical host's kernel and firmware. The guest sees a virtualized CPU and can issue power-management instructions, but what actually happens to the underlying physical core is entirely up to the hypervisor. This is one of several reasons a cloud VM's CPU behavior can differ, in subtle ways, from an identically-specced bare-metal machine — a fact worth remembering before assuming every CPU-level tuning technique in this chapter transfers identically onto virtualized infrastructure.

CPU Affinity and Pinning — Taking Manual Control#

Every mechanism covered so far — the scheduler moving threads between cores, SMT sharing execution units, NUMA's local-versus-remote memory penalty — happens under the kernel's default, general-purpose policy, which is tuned to be reasonable for the widest range of workloads, not optimal for any one specific workload. CPU affinity (also called CPU pinning) is the mechanism that lets an operator or the kernel's own resource-management layer override that default and bind a specific process or container to a specific, fixed set of cores.

What pinning actually buys: by restricting a process to a known, fixed set of cores, its memory allocations stay consistently local to one NUMA node (directly avoiding Section 9's remote-access penalty), its cache state (Section 6) is never invalidated by being migrated to a different core mid-execution, and — for the most latency-sensitive workloads — specific cores can be reserved exclusively for one process, guaranteeing it never has to context-switch with, or share SMT execution units with, anything else at all.

# Manually pin an already-running process to cores 4-7
taskset -pc 4-7 <pid>

# Confirm the kernel's current view of what a process is allowed to run on
grep Cpus_allowed_list /proc/<pid>/status
# Cpus_allowed_list:    4-7

# A Kubernetes pod spec that qualifies for exclusive, pinned cores under the
# CPU Manager 'static' policy: Guaranteed QoS class requires requests == limits,
# AND the CPU value must be a whole number (not e.g. "500m")
apiVersion: v1
kind: Pod
metadata:
  name: checkout-service-latency-critical
spec:
  containers:
    - name: checkout-service
      image: checkout-service:latest
      resources:
        requests:
          cpu: "4"      # whole number, matches limits exactly
          memory: "2Gi"
        limits:
          cpu: "4"
          memory: "2Gi"

From the Trenches: a payments team running the most latency-sensitive part of checkout-service — final transaction authorization — noticed p99 latency had a long, inconsistent tail on Kubernetes even though average CPU utilization on the node was comfortably low. The immediate cause, found by comparing Cpus_allowed_list for the pod's process against the node's actual per-core mpstat activity, was that the pod was running under Kubernetes' default CPU Manager policy (none), meaning the container's threads were free to migrate between any core on the node at the scheduler's discretion — including onto cores the node's own system daemons and other, noisier pods were also actively using. Each migration cost a real, if individually small, cache-locality penalty (Section 6), and enough of them landing in the transaction-authorization hot path was producing the tail latency. The underlying, two-levels-deep condition was that the team had sized the pod's CPU request correctly (enough vCPUs for its actual load) but had never revisited the node's CPU Manager policy itself — static versus none is a cluster-wide kubelet configuration, invisible from the pod spec alone, and nobody had connected "our pod's CPU request looks right" with "but is this node even capable of honoring exclusive pinning." The fix was moving this specific workload to a dedicated node pool running the static CPU Manager policy, with the pod spec updated to the whole-number Guaranteed-QoS shape shown above, which qualified it for genuinely exclusive, pinned cores.

Reading CPU Topology on a Real Machine#

Everything covered so far in this chapter is directly readable on any real Linux host, without any special tooling — this is deliberately the single most useful practical skill from this chapter, because it turns "how many CPUs does this machine really have, and what state is it actually in" from a vague question into a precise, answerable one.

lscpu
# Architecture:            x86_64
# Vendor ID:                GenuineIntel
# Model name:               Intel(R) Xeon(R) Platinum 8375C CPU @ 2.90GHz
# CPU(s):                  16
# On-line CPU(s) list:     0-15
# Thread(s) per core:      2      <-- SMT/Hyper-Threading is enabled (Section 4)
# Core(s) per socket:      8
# Socket(s):               1      <-- single NUMA node, no cross-socket concern here
# NUMA node(s):            1
# NUMA node0 CPU(s):       0-15
# L1d cache:                256 KiB   (8 instances, one per physical core)
# L1i cache:                256 KiB
# L2 cache:                  8 MiB    (8 instances)
# L3 cache:                 32 MiB    (1 instance, shared)

# Confirm NUMA topology explicitly on a multi-socket host
numactl --hardware
# available: 2 nodes (0-1)
# node 0 cpus: 0 1 2 3 4 5 6 7
# node 0 size: 64000 MB
# node 1 cpus: 8 9 10 11 12 13 14 15
# node 1 size: 64000 MB
# node distances:
# node   0   1
#   0:  10  21     <-- local access = 10, remote access = 21 (roughly 2.1x)
#   1:  21  10

# Watch per-category CPU time live — the breakdown Section 7 depends on
mpstat -P ALL 1
# CPU  %usr  %nice  %sys  %iowait  %irq  %soft  %steal  %idle
# all   45.2    0.0  12.1      0.3   0.1    8.4     2.1   31.8
#   0   38.0    0.0  10.2      0.1   0.2   22.5     1.9   27.1   <-- this core is softirq-heavy
#   1   52.4    0.0  13.8      0.4   0.0    1.1     2.3   30.0

# Check current vs. maximum vs. base frequency per core — Section 11 made real
cat /proc/cpuinfo | grep "MHz"
# cpu MHz         : 3612.045   <-- above base clock, Turbo Boost is currently active on this core

# On a burstable cloud instance, check the CPU credit balance directly
# (AWS example — the CloudWatch metric is CPUCreditBalance, queryable via the CLI)
aws cloudwatch get-metric-statistics \
  --namespace AWS/EC2 --metric-name CPUCreditBalance \
  --dimensions Name=InstanceId,Value=i-0123456789abcdef0 \
  --start-time 2026-01-01T00:00:00Z --end-time 2026-01-01T01:00:00Z \
  --period 300 --statistics Average

Between these commands, a platform engineer can answer, on any host, in well under a minute: how many physical cores versus logical threads it actually has, whether SMT is enabled, whether NUMA is a real concern on this specific machine, its full cache hierarchy sizes, exactly which category of work is consuming CPU time on each individual core right now, whether Turbo Boost is currently active, and — on a burstable cloud instance — how much runway is left before hard throttling kicks in.

A realistic worked triage, tying the whole chapter together: an alert fires for elevated p99 latency on one checkout-service replica, while its siblings on identically-sized instances look normal. The triage sequence a platform engineer familiar with this chapter would actually run, in order:

1. mpstat -P ALL 1 on the affected host — is the aggregate utilization actually high, and if so, which category (%us/%sy/%si/%wa/%st) is driving it? Suppose this shows a normal %us, but a surprisingly high %st (steal time) on every core.
2. %st (steal time) specifically means the underlying hypervisor is not giving this VM's vCPUs as much real physical CPU time as it's entitled to — a signal that belongs to Part 5's virtualization coverage, but is diagnosed with exactly this command from this chapter.
3. Cross-check against the cloud provider's own host-level metrics (where available) or simply against sibling instances on the same instance family — if only one specific host is affected, that points toward a genuinely noisy physical neighbor on that specific piece of hardware, not a systemic instance-family problem.
4. The remediation in this scenario isn't a code change at all — it's requesting the cloud provider stop/start the instance (which frequently migrates it to different underlying physical hardware) or, for a recurring pattern, moving that workload to a dedicated/isolated instance type that doesn't share physical cores with other tenants.

This is deliberately the kind of investigation that looks like "just infrastructure noise" without this chapter's vocabulary, and looks like a precise, addressable finding with it.

CPU-Bound vs. I/O-Bound Workloads — Classifying What You're Actually Running#

Several sections in this chapter have leaned on a distinction without fully naming it: whether a piece of work spends most of its time actively computing, or mostly waiting for something else to finish. Naming it explicitly, and learning to classify a real workload correctly, ties together nearly everything covered so far into one practical, everyday judgment call.

A CPU-bound task spends the overwhelming majority of its time actively executing instructions — its speed is limited by how fast the core can compute, not by anything it's waiting on. Video encoding, cryptographic hashing, and checkout-service's discount-eligibility rule evaluation from Section 5 are all CPU-bound: give them a faster core or more parallel cores, and they genuinely finish sooner. An I/O-bound task, by contrast, spends most of its time blocked — waiting on a database query, a downstream HTTP call, or a disk read to complete — during which the CPU it was "using" sits idle, available for the scheduler to hand to something else entirely. inventory-service's stock-reservation endpoint from Section 10, dominated by a database round-trip, is fundamentally I/O-bound even though it does perform some real computation along the way.

What to notice: the two quadrants at the extremes call for opposite scaling strategies. A workload in the CPU-bound quadrant benefits from more physical cores and faster clock speed, and gets no real benefit from more concurrent threads past the physical core count (Section 8's context-switching cost, Section 4's SMT contention). A workload in the I/O-bound quadrant barely touches the CPU per unit of work, so it benefits enormously from higher concurrency — many threads or async tasks can all be "in flight," blocked, at once, without meaningfully contending for CPU time — but gains almost nothing from a faster or more powerful CPU.

From the Trenches: a team building a new async task queue for catalog-service sized every worker pool identically — one config value, worker_count, applied uniformly to the image-thumbnail-generation queue and the send-order-confirmation-email queue alike, reasoning that "workers are workers." The thumbnail queue was chronically backlogged under load despite low reported CPU utilization on the host; the email queue ran fine with far fewer workers than it had been allocated, and adding more to it did nothing. The immediate cause was that thumbnail generation is genuinely CPU-bound (decoding and re-encoding image data is real, sustained computation) while sending a confirmation email is almost entirely I/O-bound (a network call to a third-party email API, followed by waiting). The underlying, two-levels-deep condition was that the shared worker-pool abstraction had been built to treat "a task" as a generic, undifferentiated unit of work, when the two task types actually had fundamentally opposite scaling behavior — the thumbnail queue needed a worker count close to the physical core count (more just added context-switch overhead per Section 8, with no throughput gain, exactly matching the low-CPU-utilization symptom since workers spent that time switching rather than computing), while the email queue could have used a much larger pool sized for its concurrency, not its CPU demand. The fix was splitting the single generic worker pool into two independently sized pools, one per task category, using this exact classification.

Interview-ready line: "A CPU-bound task spends most of its time actively computing, so it scales with more cores and faster clock speed but gets little benefit from more concurrent threads past the physical core count. An I/O-bound task spends most of its time blocked waiting on something external, so it scales with more concurrency — many threads can be in flight at once without meaningfully contending for CPU — but barely benefits from a faster CPU at all. Sizing a thread pool or worker count correctly means classifying the work first, not applying one default to everything."

How These Concepts Show Up on the Cloud Bill#

Every mechanism covered in this chapter has a direct, often underappreciated cost-optimization angle — this is deliberately a closing synthesis, not a new topic, connecting hardware-level concepts back to a conversation most platform teams actually have every quarter.

Common Mistakes and Interview Traps#

Worked Practice Problems#

Problem 1: A host's lscpu output shows Thread(s) per core: 2 and Core(s) per socket: 8, with a single socket. A monitoring dashboard reports "16 CPUs at 90% utilization" during a load test, and the on-call engineer concludes the host needs to be resized to more vCPUs. What question should be asked before accepting that conclusion, and why?

Answer: The first question should be: what is the category breakdown of that 90% utilization (%us/%sy/%si/%wa/%st via mpstat), and is the workload genuinely saturating all 8 physical cores, or is it two threads per physical core contending for the same shared execution units (Section 4)? With SMT enabled, 16 "CPUs" is 8 real cores presented as 16 logical threads — a workload that's CPU-bound and running one thread per logical CPU may already be near the physical ceiling of what those 8 cores can deliver, in which case resizing to genuinely more physical cores helps, but doubling the vCPU count on the same physical-core-to-thread ratio would not deliver anywhere near double the real throughput. The utilization number alone can't distinguish "needs more real cores" from "already near the SMT-contention ceiling of the cores it has."

Problem 2: checkout-service runs on a two-socket host. After a routine host reboot (which can reset process-to-core affinity), p99 latency on one specific replica becomes noticeably worse than its siblings on identical hardware, with no corresponding change in request volume, error rate, or code version. Walk through the two most likely CPU-architecture-level explanations from this chapter, and how you'd distinguish between them.

Answer: The two most likely explanations are (1) the process ended up pinned to cores on a different NUMA node than the memory it originally allocated, paying a remote-access penalty on every memory read (the NUMA section), or (2) the process's interrupt-heavy network processing ended up sharing a core with — or being scheduled onto — cores handling a disproportionate share of softirq/interrupt load (the interrupts section), leaving less real CPU time for request handling than its siblings have. Distinguishing them: numastat -p <pid> reveals a high numa_miss ratio for explanation (1); mpstat -P ALL showing one specific core with an outsized %soft share, cross-referenced with which core the process's threads are actually running on (taskset -pc <pid> or /proc/<pid>/status's Cpus_allowed field), reveals explanation (2). Both are entirely invisible to standard application-level latency dashboards and require exactly this kind of hardware-topology-aware investigation.

Problem 3: During a design review, a teammate proposes reducing inventory-service's application-level thread pool from 200 threads down to 16 on an 8-core (16 logical thread, SMT-enabled) host, arguing "we have 16 logical CPUs, so 16 threads should be the right number." Is this reasoning sound, and what's missing from it?

Answer: It's the right instinct but an incomplete rule — 16 is a reasonable starting point for the purely CPU-bound portion of the workload specifically because it roughly matches the logical thread count, but it ignores that inventory-service, like most real services, is a mix of CPU-bound work (business logic, serialization) and I/O-bound work (database calls, downstream HTTP calls) that spends most of its time blocked, not computing. Sizing the entire thread pool to the logical CPU count would starve the I/O-bound portion of the workload of enough concurrency to keep multiple in-flight requests moving while others wait on network calls. The more complete fix, covered further in Part 2 and Part 7 of this series, is separating CPU-bound and I/O-bound work into differently-sized pools (or using a runtime with async I/O that doesn't consume a blocked OS thread at all) rather than picking one thread-pool number for the whole service.

Problem 4: A cost-optimization proposal suggests moving catalog-service's stateless web tier — currently on fixed-performance instances at roughly 35% average CPU utilization with occasional short spikes to 70% — onto a burstable (T-series-style) instance family to save cost. Is this a good fit for burstable instances, and what would you check before approving it?

Answer: This actually is a reasonable candidate profile for burstable instances — low sustained baseline utilization with occasional short bursts is exactly the usage pattern the credit-accumulation model is designed for (Section 11): the workload spends most of its time below baseline, earning credits, and spends them during the brief spikes. What to check before approving it: (1) confirm the spikes are genuinely short relative to the instance family's credit-earn rate at low utilization — a spike that's short but frequent enough could still net-drain credits over time even if each individual spike looks brief; (2) load-test a realistic worst-case traffic pattern (not just a short synthetic burst) long enough to see whether the credit balance would actually stay positive under real sustained peak-hour traffic, not just a lab benchmark; and (3) set a CloudWatch alarm (or provider equivalent) on the CPU credit balance metric itself, so a genuine baseline-usage regression is caught as a proactive signal rather than discovered as a sudden, confusing throttling incident in production.

Problem 5: inventory-service's stock-reservation endpoint was scaled from 4 to 16 cores and only achieved a 1.4x throughput improvement (the same scenario as Section 10's From the Trenches example). A teammate proposes scaling to 64 cores next, reasoning "if 4x cores got 1.4x, 16x cores should get proportionally more." Using Amdahl's Law, explain why this reasoning is likely to make the situation worse, not better, without a code change first.

Answer: The 1.4x result at 4x the cores already reveals that the workload's serial fraction (the database row-lock identified in Section 10's investigation) is large relative to its parallel fraction — plugging the observed numbers into Amdahl's Law's speedup formula backs out an implied parallel fraction well under 50%, which means the theoretical maximum speedup even with infinite cores is well under 2x. Scaling further, to 64 cores, would not "unlock more proportional gains" — it would push further along the same flattening curve shown in Section 10's diagram, delivering a shrinking marginal return for a 4x larger hardware spend, and would very likely make the serial bottleneck itself worse: more concurrent threads all competing for the same single row-level lock increases lock contention and wait time, a dynamic the classic Amdahl formula doesn't even capture (it assumes the serial fraction's duration is fixed, when in practice contention on a shared lock tends to grow with the number of contending threads). The correct next step, following Section 10's own resolution, is addressing the lock contention directly — narrowing the lock's scope or batching writes — which raises the ceiling for every future core count, rather than spending more on hardware against a bottleneck that hardware cannot fix.

Problem 6: checkout-service sees traffic in sharp, unpredictable bursts around flash-sale announcements, with long idle stretches between them. After one such announcement, the very first handful of requests in the burst show noticeably higher latency than requests just a few seconds later in the same burst, even though the host's CPU utilization was near zero immediately beforehand. What two mechanisms from this chapter are the most likely explanations, and how would you distinguish them?

Answer: The two most likely explanations are (1) the CPU cores handling the request had dropped into a deep idle C-state during the preceding quiet period, and paid real wake-up latency on the first requests of the burst before returning to full speed (Section 12), and (2) if running on a burstable cloud instance family, the host may still be below its CPU credit baseline from the idle period and hasn't yet ramped to full burst frequency, though this is a less likely first-request-specific cause since credit-based throttling affects sustained performance more than the very first moment of a burst. Distinguishing them: checking whether the host is a burstable instance family at all (Section 11) rules explanation (2) in or out immediately; for explanation (1), comparing cpupower frequency-info's live frequency reading during the very first requests against a steady-state reading a few seconds into the same burst would show a core still transitioning up from a deep idle state. In practice, for a workload with this exact bursty-with-long-idle-gaps profile, restricting maximum idle-state depth (Section 12's tip) is the more targeted fix than avoiding burstable instances altogether, since the C-state wake latency is what's actually producing the specific "first few requests only" pattern described.

Summary and What's Next#

Quick reference — key terms from this chapter:

A CPU core is not a single, simple thing that either has "enough" or "not enough" capacity — it's a hierarchy of cores, logical threads sharing physical execution units, cache layers with dramatically different latencies, an instruction set architecture that increasingly comes in more than one real cloud-relevant flavor, and (on multi-socket hosts) a NUMA topology that makes even "memory" not a single uniform resource. Interrupts and context switches are the mechanisms that let one core appear to run many things at once, dynamic frequency scaling and credit-based throttling both mean "how fast is this core right now" is a moving target rather than a fixed spec-sheet number, and every one of these concepts is directly, cheaply observable on a real host with lscpu, numactl, mpstat, and perf stat — tools worth knowing before the next unexplained latency graph shows up.

Part 2 builds directly on this foundation: the CPU concepts covered here (cores, context switches, cache locality) are exactly what the Linux scheduler has to reason about every time it decides which thread runs on which core, for how long. Part 2 covers the scheduler itself — the Completely Fair Scheduler's fairness algorithm and its EEVDF successor (the default since Linux 6.6), nice values and priorities, cgroup CPU quotas (the mechanism behind every Kubernetes CPU limit), and how to read top's load average correctly, which turns out to be one of the most commonly misunderstood numbers in all of systems operations.