1.5 — The CPU

Everything so far has been parts. 1.1 gave switches and gates; 1.2 gave adders, registers, and the finite state machine; 1.3 and 1.4 taught the bits to mean numbers, text, and pictures. This chapter assembles them into the thing that runs the world: the CPU (central processing unit), the "brain" that does one thing after another, very fast.

The plan: understand the one architectural idea that makes a general-purpose computer possible (Von Neumann), watch the CPU's heartbeat cycle (fetch-decode-execute), then do the thing this whole Part has been building toward — take a single line of code, sum(a, b), and follow it all the way down to the transistors. After that, the three tricks modern CPUs use to go fast — pipelining, branch prediction, and speculation — and the spectacular security disaster the last one caused.

1. The stored-program idea — Von Neumann's leap

Early computers were rewired to change what they did — ENIAC was reprogrammed by physically replugging cables for days. The breakthrough, credited to John von Neumann's 1945 report (building on Turing's theory and the work of Eckert and Mauchly), was deceptively simple: store the program in the same memory as the data. Instructions become just numbers in memory, and the machine reads them one after another. Change the numbers, change the behavior — no rewiring. Every mainstream computer since is a Von Neumann architecture: a CPU connected to one memory holding both code and data, shuttling information back and forth across a bus.

Figure 1 — Von Neumann architecture. One memory holds both instructions and data. The control unit (a finite state machine from 1.2) fetches instructions; the ALU computes; registers hold the working values. Everything moves over the bus.

Look what's inside, and notice you already built all of it: the ALU (arithmetic-logic unit) is the combinational adder/logic block from 1.2; the registers are rows of flip-flops from 1.2; the control unit is a finite state machine. Two registers are special. The program counter (PC) holds the memory address of the next instruction — it's the machine's finger tracking its place. The instruction register (IR) holds the instruction currently being executed. The CPU is, quite literally, the finite state machine of 1.2 wearing a crown: its states step an endless cycle.

2. Fetch–decode–execute — the heartbeat

The CPU does the same three steps forever, once per instruction, driven by the clock:

1. Fetch: read the instruction at the address in the PC from memory into the IR. Advance the PC to the next instruction.
2. Decode: the control unit interprets the instruction's bits — what operation? which registers? which memory address?
3. Execute: perform it — add in the ALU, load/store to memory, or jump (overwrite the PC to go somewhere else).

Then repeat, billions of times a second. That's the entire soul of a processor. Loops, function calls, if statements, this web page, your operating system — all of it is this cycle, running fast enough to feel like magic. An if is just an instruction that conditionally changes the PC; a loop is an instruction that sets the PC backwards.

But the CPU doesn't understand sum(a, b). It understands only machine code — numbers, each a small operation from its fixed vocabulary, the instruction set architecture (ISA). The ISA is the contract between software and silicon: x86-64 (Intel/AMD, in most laptops and servers), ARM (in your phone and Apple Silicon), and the open RISC-V are the ones that matter today. To make machine code human-readable we write assembly, a one-to-one text form (e.g. add rax, rbx means "add register rbx into rax"). So there's a translation chain from your code down to the metal — and now we walk it, end to end.

3. The flagship trace: sum(a, b) from source to silicon

Here is a function in a C-like language. We will follow it all the way down.

int sum(int a, int b) {
    return a + b;
}
// called somewhere as: sum(5, 7)

Level 1 — Source code. Human intent: "combine two integers." Readable, but meaningless to a CPU. A compiler (Part 3) translates it toward the machine, and to do so it must decide where the values live. By the platform's calling convention (an agreed contract for how functions pass arguments), the first two integer arguments arrive in specific registers — on x86-64 Linux, edi and esi. The result must come back in eax.

Level 2 — Assembly. The compiler emits something close to this (simplified, x86-64):

sum:
    mov   eax, edi     ; copy 1st argument (a = 5) into eax
    add   eax, esi     ; add 2nd argument (b = 7) into eax  → eax = 12
    ret                ; return; result is in eax

Three instructions. mov copies, add adds, ret returns to the caller. Each corresponds to one CPU operation. Notice the whole function is register arithmetic — no memory needed — which is why it's blindingly fast.

Level 3 — Machine code. The assembler turns each line into the actual bytes the ISA defines. add eax, esi becomes the bytes 01 F0. Here 01 is the opcode — the numeric code that identifies which operation to perform (the ISA's dictionary assigns "add register to register" the number 01) — and F0 is the operand byte encoding which registers to use ("source esi, destination eax"). Now it's pure numbers in memory — the stored program of Von Neumann, indistinguishable from data until the PC points at it.

Figure 2 — The translation stack. One line of source becomes a few assembly instructions, then a handful of bytes in memory. The CPU only ever sees the bottom layer.

Level 4 — Execution, cycle by cycle. Now the CPU runs add eax, esi (with 5 already in eax, 7 in esi):

• Fetch: the PC holds this instruction's address; the CPU reads the bytes 01 F0 from memory into the IR. The PC advances past them.
• Decode: the control unit (the FSM) reads 01 F0: operation = ADD, source = esi, destination = eax. It sets up the datapath — routing the two registers' outputs into the ALU's inputs.
• Execute: the register file presents 5 and 7 to the ALU, which is the ripple/lookahead adder you built in 1.2. Its gates settle to 00000000...00001100 = 12. On the clock edge, that result is latched back into the eax register.

Level 5 — The physics. And what is the ALU actually doing? The XOR and AND gates of the half-adders from 1.2, which are themselves CMOS transistor-switches from 1.1, which are voltages crossing noise margins. The "12" that lands in eax is a pattern of high and low voltages held by flip-flops — charge sitting in microscopic buckets. 5 + 7 = 12 has become electrons finding a stable configuration in silicon, in well under a nanosecond.

That is the whole stack, unbroken: intent → source → assembly → bytes → cycle → gates → transistors → voltage. Every program you will ever write rides this elevator down. When someone says a computer "just moves electricity around," this trace is what they mean — and now you can defend every rung of it. ^{⚑Trace a simple sum() from source code down to low-level machine/microprocessor execution, end to end. [EQ-74]}

4. Going fast — pipelining, prediction, speculation

A naïve CPU does fetch, then decode, then execute, then starts the next instruction — four steps, one at a time, most of the chip idle at any moment. Three ideas fix that, and they define modern performance.

Pipelining. Like a car assembly line: while instruction #1 is being executed, #2 is being decoded, and #3 is being fetched — all at once, in different stages of the chip. The pipeline doesn't make one instruction finish faster; it makes many instructions overlap, so throughput can approach one finished instruction per clock cycle. Real CPUs have 15–20+ pipeline stages. (Remember the critical path from 1.2? Pipelining is why chips break work into short stages — shorter stages, higher clock.)

The branch problem — and prediction. Pipelining assumes the CPU knows which instruction comes next. But an if (a conditional branch) doesn't resolve until it executes — and by then the CPU has already fetched and started the instructions right after it, guessing. Guess wrong and it must throw away that work (a "pipeline flush") — expensive. So CPUs contain a branch predictor, a small machine that learns each branch's history ("this loop condition has been true 999 times, bet it's true again") and predicts which way to go, achieving well over 95% accuracy. It fetches and starts running down the predicted path before knowing if it's right.

Speculative execution. That last move is speculation: the CPU runs ahead on the predicted path, and if the guess was right (usually), the work is already done — a big speedup. If wrong, it discards the speculative results and no architectural harm is done. This is one of the largest performance wins in modern computing.

5. The expert lens: when speculation broke security (Spectre)

For decades, "if wrong, discard the speculative results and no harm done" was accepted as obviously true. In 2018 it was shown to be false, in one of the most important security findings in the field's history — Spectre and Meltdown.

The flaw: when the CPU discards mis-speculated work, it correctly rolls back the registers and memory — but it does not roll back the cache (1.6). Data the speculative path touched is left sitting in the fast cache. An attacker can trick the predictor into speculatively reading a secret it isn't allowed to (a password, another program's memory), and although the CPU discards the read, the secret has now influenced which memory is cached. The attacker then measures memory access timing — cached data returns faster — and reads the secret back out through that side channel, one bit at a time. The security boundary was never crossed architecturally; it was crossed in timing.

This broke the fundamental isolation assumption of shared computers — one program reading another's secrets, across the walls the OS enforces — and forced patches into virtually every CPU, operating system, and cloud on Earth, some costing real performance. The lesson is profound and worth carrying into every system you design: an abstraction can be perfectly correct in its stated model and still leak through a channel the model ignored — here, time. Performance optimizations can have security consequences invisible at the level you're reasoning at. You'll meet side channels again in cryptography (Part 8); their birthplace is right here, in the CPU's hunger for speed.

Next chapter: the trace above quietly assumed the CPU could get 5 and 7 instantly. It can't — memory is far slower than the CPU, and the gap is the single biggest factor in real-world performance. 1.6 builds the cache hierarchy that hides it, and shows why "cache-friendly" code can run 10× faster with identical logic.

Recall

• Von Neumann architecture: program and data share one memory; instructions are just numbers the CPU reads in sequence. The ALU, registers, and control unit are the adder, flip-flops, and FSM of 1.2.
• The CPU repeats fetch → decode → execute forever; the program counter tracks the next instruction. It understands only machine code from its ISA (x86-64, ARM, RISC-V); assembly is its readable form.
• The flagship trace: a + b → add eax, esi → bytes 01 F0 → the ALU's gates settling → transistors switching → voltages. Intent to electrons, unbroken.
• Speed comes from pipelining (overlap instructions), branch prediction (guess the next path >95% right), and speculative execution (run ahead on the guess).
• Spectre/Meltdown: speculation rolls back registers but not the cache, leaking secrets through a timing side channel — a correct abstraction breached through a channel (time) its model ignored.

Self-test: What made the stored-program idea revolutionary? Name the three cycle steps and what the PC does. Trace add eax, esi through fetch-decode-execute. Why does branch misprediction cost performance? In one sentence, how does Spectre steal a secret it never "officially" read?

Quiz Bank

FoundationalWhat is the Von Neumann architecture and why did it matter?

A design where a single memory stores both instructions and data, and the CPU reads instructions from it sequentially. It mattered because it made computers general-purpose and reprogrammable by loading new numbers instead of physical rewiring — instructions became data. Its components map directly to earlier chapters: ALU (combinational adder), registers (flip-flops), control unit (finite state machine).

FoundationalDescribe the fetch-decode-execute cycle.

Fetch: read the instruction at the program counter's address into the instruction register, and advance the PC. Decode: the control unit interprets the bits — operation, operands, addressing. Execute: carry it out (ALU operation, memory load/store, or a jump that rewrites the PC). Repeat every cycle. Control flow is just instructions that conditionally or unconditionally change the PC.

AppliedTrace what happens when the CPU runs `add eax, esi`.

Fetch: the bytes (01 F0) at the PC are loaded into the IR; PC advances. Decode: the FSM reads them as "ADD, source esi, destination eax" and routes those registers into the ALU. Execute: the ALU — the adder built from XOR/AND gates (1.2) of CMOS transistors (1.1) — computes the sum, and on the clock edge it's latched into eax. The abstract a + b becomes gates settling and voltages held in flip-flops.

AppliedWhat is an ISA, and name the three that matter today.

The instruction set architecture is the contract between software and hardware: the exact vocabulary of machine instructions a CPU understands, their encodings, and register model. Code compiled for one ISA won't run on another. The three current ones: x86-64 (Intel/AMD; PCs, servers), ARM (phones, Apple Silicon, increasingly servers — power-efficient), and RISC-V (open, royalty-free, rising).

InterviewWhat is pipelining and what limits its benefit?

Pipelining overlaps instruction stages (fetch/decode/execute/…) so multiple instructions are in flight at once, raising throughput toward one instruction per cycle without speeding up any single one. Limits: hazards — data hazards (an instruction needs a result not yet ready), and control hazards (branches, whose direction isn't known when the next fetch must happen). Branches are the big one, which is why CPUs add branch prediction and speculation; a misprediction flushes the pipeline, wasting the partially-done work.

InterviewWhy does branch misprediction hurt, and how do CPUs reduce it?

A pipelined CPU must fetch instructions after a branch before the branch resolves, so it guesses the direction and starts executing speculatively. A wrong guess means all that speculative work is discarded (a pipeline flush), and the pipeline refills from the correct path — costing many cycles. CPUs reduce it with a branch predictor that learns each branch's history and achieves >95% accuracy; well-predicted branches are nearly free, while unpredictable ones (e.g. data-dependent, random) are costly — which is why sorted data can make identical code run faster.

StaffExplain Spectre to a fellow engineer and state its general lesson for system design.

Modern CPUs speculatively execute past branches on predicted paths, discarding results if wrong. Spectre exploits that the discard rolls back registers/memory but not the cache: an attacker mistrains the branch predictor so the CPU speculatively reads a forbidden secret and uses it to index memory, leaving a footprint in the cache. Even though the read is "discarded," the attacker recovers the secret by timing memory accesses (cached = faster) — a side channel. General lesson: correctness in an abstraction's stated model doesn't guarantee security, because real implementations leak through channels the model ignores (here, time). Performance optimizations can silently create security boundaries you weren't reasoning about — a mindset you carry into caching, crypto, and multi-tenant systems.

Flashcards

FlashStored-program (Von Neumann) idea

Instructions and data live in the same memory; code is just numbers the CPU reads in order — reprogram by changing memory, not wiring.

FlashThe three cycle steps

Fetch → Decode → Execute, repeated every clock cycle.

FlashProgram counter (PC)

Register holding the address of the next instruction; jumps/branches/loops work by changing it.

FlashISA — three that matter

x86-64 (Intel/AMD), ARM (mobile/Apple), RISC-V (open).

FlashPipelining vs speculation

Pipelining overlaps instruction stages for throughput; speculation runs ahead on a predicted branch before knowing it's correct.

FlashSpectre in one line

Speculative execution leaves secrets in the cache; timing side-channel reads them out despite the speculation being "discarded."

Scenario Drill

DrillTwo versions of a loop do identical work over an array of numbers, but one runs several times faster — the only difference is that the fast one's data is sorted first. Using this chapter, explain why.

The loop almost certainly contains a data-dependent branch (e.g. if (value > threshold) …). On unsorted data the branch outcome is effectively random, so the branch predictor is wrong about half the time; each misprediction flushes the pipeline and wastes many cycles. On sorted data the branch is false for a long run and then true for a long run — highly predictable — so the predictor is right nearly always and speculation pays off, keeping the pipeline full. Identical logic, but the CPU's ability to guess the next instruction changed completely. (This is the famous "why is processing a sorted array faster" result.) The engineering takeaways: unpredictable branches are a real cost, and sometimes branchless code — computing both sides and selecting — beats a mispredicted branch.