1.5 — The CPU

Everything so far has been parts. 1.1 gave switches and gates; 1.2 gave adders, registers, and the finite state machine; 1.3 and 1.4 taught the bits to mean numbers, text, and pictures. This chapter assembles them into the thing that runs the world: the CPU (central processing unit), the "brain" that does one thing after another, very fast.

The plan: understand the one architectural idea that makes a general-purpose computer possible (Von Neumann), watch the CPU's heartbeat cycle (fetch-decode-execute), then do the thing this whole Part has been building toward — take a single line of code, sum(a, b), and follow it all the way down to the transistors. After that, the three tricks modern CPUs use to go fast — pipelining, branch prediction, and speculation — and the spectacular security disaster the last one caused.

1. The stored-program idea — Von Neumann's leap

Early computers were rewired to change what they did — ENIAC was reprogrammed by physically replugging cables for days. The breakthrough, credited to John von Neumann's 1945 report (building on Turing's theory and the work of Eckert and Mauchly), was deceptively simple: store the program in the same memory as the data. Instructions become just numbers in memory, and the machine reads them one after another. Change the numbers, change the behavior — no rewiring. Every mainstream computer since is a Von Neumann architecture: a CPU connected to one memory holding both code and data, shuttling information back and forth across a bus.

Figure 1 — Von Neumann architecture. One memory holds both instructions and data. The control unit (a finite state machine from 1.2) fetches instructions; the ALU computes; registers hold the working values. Everything moves over the bus.

Look what's inside, and notice you already built all of it: the ALU (arithmetic-logic unit) is the combinational adder/logic block from 1.2; the registers are rows of flip-flops from 1.2; the control unit is a finite state machine. Two registers are special. The program counter (PC) holds the memory address of the next instruction — it's the machine's finger tracking its place. The instruction register (IR) holds the instruction currently being executed. The CPU is, quite literally, the finite state machine of 1.2 wearing a crown: its states step an endless cycle.

2. Fetch–decode–execute — the heartbeat

The CPU does the same three steps forever, once per instruction, driven by the clock:

1. Fetch: read the instruction at the address in the PC from memory into the IR. Advance the PC to the next instruction.
2. Decode: the control unit interprets the instruction's bits — what operation? which registers? which memory address?
3. Execute: perform it — add in the ALU, load/store to memory, or jump (overwrite the PC to go somewhere else).

Then repeat, billions of times a second. That's the entire soul of a processor. Loops, function calls, if statements, this web page, your operating system — all of it is this cycle, running fast enough to feel like magic. An if is just an instruction that conditionally changes the PC; a loop is an instruction that sets the PC backwards.

But the CPU doesn't understand sum(a, b). It understands only machine code — numbers, each a small operation from its fixed vocabulary, the instruction set architecture (ISA). The ISA is the contract between software and silicon: x86-64 (Intel/AMD, in most laptops and servers), ARM (in your phone and Apple Silicon), and the open RISC-V are the ones that matter today.

Why are there three different sets of rules, and why do they live in different devices? It comes down to power and permission.

Think of x86-64 (named as the 64-bit extension of Intel's original 8086 processor line) as a massive, heavy-duty toolbox. It was created by Intel decades ago for desktop computers. Over time, Intel added more and more complex instructions to its vocabulary so it could do heavy work in fewer steps. Because it has such a huge vocabulary, an x86-64 CPU requires a lot of physical circuitry just to figure out what each instruction means. More circuitry means it draws more electricity, and more electricity means it gets hot. That is fine for a server or a thick laptop plugged into a wall, but it is terrible for a battery-powered phone in your pocket.

ARM (which stands for Advanced RISC Machines, where RISC means Reduced Instruction Set Computer) was built to solve the battery problem. Instead of a massive toolbox, ARM uses a smaller, simpler set of instructions. Because the vocabulary is smaller, the CPU needs less circuitry to decode the instructions. Less circuitry means it uses far less electricity and stays cool. To do heavy work, an ARM chip just strings a lot of these simple instructions together very fast. This power efficiency is why ARM dominates every smartphone on Earth. Recently, companies like Apple have proved that this simple approach can be scaled up to run fast laptops and desktops too, which is exactly what Apple Silicon does.

RISC-V (pronounced "RISC-five", standing for the fifth generation of the open Reduced Instruction Set Computer standard) solves a problem of ownership rather than physics. If you want to design and sell an x86-64 chip, you must get a legal license from Intel or AMD. If you want to build an ARM chip, you must pay the company that owns ARM a massive fee. You are not allowed to just use their vocabulary. RISC-V is different. It is an open, free rulebook. Anyone can download the RISC-V vocabulary and build a physical chip that understands it, without asking permission or paying anyone a cent. Because it is free, it is rapidly becoming the standard for custom hardware, from smart home gadgets to specialized data center chips.

To make machine code human-readable we write assembly, a one-to-one text form (e.g. add rax, rbx means "add register rbx into rax"). So there's a translation chain from your code down to the metal — and now we walk it, end to end.

3. The flagship trace: sum(a, b) from source to silicon

Here is a function in a C-like language. We will follow it all the way down.

int sum(int a, int b) {
    return a + b;
}
// called somewhere as: sum(5, 7)

Level 1 — Source code. Human intent: "combine two integers." Readable, but meaningless to a CPU. A compiler (Part 3) translates it toward the machine, and to do so it must decide where the values live. By the platform's calling convention (an agreed contract for how functions pass arguments), the first two integer arguments arrive in specific registers — on x86-64 Linux, edi and esi. The result must come back in eax.

Level 2 — Assembly. The compiler emits something close to this (simplified, x86-64):

sum:
    mov   eax, edi     ; copy 1st argument (a = 5) into eax
    add   eax, esi     ; add 2nd argument (b = 7) into eax  → eax = 12
    ret                ; return; result is in eax

Three instructions. mov copies, add adds, ret returns to the caller. Each corresponds to one CPU operation. Notice the whole function is register arithmetic — no memory needed — which is why it's blindingly fast.

Level 3 — Machine code. The assembler turns each line into the actual bytes the ISA defines. add eax, esi becomes the bytes 01 F0. Here 01 is the opcode — the numeric code that identifies which operation to perform (the ISA's dictionary assigns "add register to register" the number 01) — and F0 is the operand byte encoding which registers to use ("source esi, destination eax"). Now it's pure numbers in memory — the stored program of Von Neumann, indistinguishable from data until the PC points at it.

Figure 2 — The translation stack. One line of source becomes a few assembly instructions, then a handful of bytes in memory. The CPU only ever sees the bottom layer.

Level 4 — Execution, cycle by cycle. Now the CPU runs add eax, esi (with 5 already in eax, 7 in esi):

• Fetch: the PC holds this instruction's address; the CPU reads the bytes 01 F0 from memory into the IR. The PC advances past them.
• Decode: the control unit (the FSM) reads 01 F0: operation = ADD, source = esi, destination = eax. It sets up the datapath — routing the two registers' outputs into the ALU's inputs.
• Execute: the register file presents 5 and 7 to the ALU, which is the ripple/lookahead adder you built in 1.2. Its gates settle to 00000000...00001100 = 12. On the clock edge, that result is latched back into the eax register.

Level 5 — The physics. And what is the ALU actually doing? The XOR and AND gates of the half-adders from 1.2, which are themselves CMOS transistor-switches from 1.1, which are voltages crossing noise margins. The "12" that lands in eax is a pattern of high and low voltages held by flip-flops — charge sitting in microscopic buckets. 5 + 7 = 12 has become electrons finding a stable configuration in silicon, in well under a nanosecond.

That is the whole stack, unbroken: intent → source → assembly → bytes → cycle → gates → transistors → voltage. Every program you will ever write rides this elevator down. When someone says a computer "just moves electricity around," this trace is what they mean — and now you can defend every rung of it. ^{⚑Trace a simple sum() from source code down to low-level machine/microprocessor execution, end to end. [EQ-74]}

4. Going fast — pipelining, prediction, speculation

A naïve CPU does fetch, then decode, then execute, then starts the next instruction — four steps, one at a time, most of the chip idle at any moment. Three ideas fix that, and they define modern performance.

Pipelining. Like a car assembly line: while instruction #1 is being executed, #2 is being decoded, and #3 is being fetched — all at once, in different stages of the chip. The pipeline doesn't make one instruction finish faster; it makes many instructions overlap, so throughput can approach one finished instruction per clock cycle. Real CPUs have 15–20+ pipeline stages. (Remember the critical path from 1.2? Pipelining is why chips break work into short stages — shorter stages, higher clock.)

Think of doing laundry. If you wash, dry, and fold one load before starting the next, your washer and dryer sit empty most of the time. That is the naïve CPU. Instead, you should start washing the second load the moment the first load goes into the dryer. You are now doing three loads of laundry at the exact same time, just in different stages. It still takes the same amount of time for one specific shirt to get clean. However, you are finishing a whole basket of laundry much faster. A modern CPU does this with code. It slices the work into up to 20 tiny steps, keeping every part of the chip busy all the time.

Figure 3 — Pipelining. While Instruction 3 is being fetched, Instruction 2 is decoding, and Instruction 1 is executing. The chip finishes one instruction every cycle instead of every three cycles.

The branch problem — and prediction. Pipelining assumes the CPU knows which instruction comes next. But an if (a conditional branch) doesn't resolve until it executes — and by then the CPU has already fetched and started the instructions right after it, guessing. Guess wrong and it must throw away that work (a "pipeline flush") — expensive. So CPUs contain a branch predictor, a small machine that learns each branch's history ("this loop condition has been true 999 times, bet it's true again") and predicts which way to go, achieving well over 95% accuracy. It fetches and starts running down the predicted path before knowing if it's right.

But what happens when the road splits? Imagine driving a fast train on a track that branches left or right. You only find out which way you need to go when you actually reach the junction. If you stop the train to wait for the signal to change, you waste time. If you guess and go left, but the signal eventually says right, you have to stop, back the train all the way up, and switch tracks. That painful reversing is a pipeline flush. Stopping and backing up destroys your speed. To solve this, the CPU remembers every junction it has ever seen. If you turned left the last hundred times at a specific split, the CPU confidently steers the train left again without slowing down. Because code is repetitive, it is almost always right.

To see what this junction looks like in real code, consider an order pipeline checking if items are in stock:

for (let item of orders) {
    // (1) The branch: CPU must guess true or false here
    if (item.inStock) {
        shipOrder(item);
    }
}

When the CPU reaches // (1), the condition is a fork in the road, but the pipeline cannot pause to evaluate it. If the last hundred items were in stock, the branch predictor simply guesses that the next one is too. It confidently steers the execution down the shipOrder path without dropping its speed.

Speculative execution. That last move is speculation: the CPU runs ahead on the predicted path, and if the guess was right (usually), the work is already done — a big speedup. If wrong, it discards the speculative results and no architectural harm is done. This is one of the largest performance wins in modern computing.

Speculation is the act of actually doing the work before you know you need it. Imagine a chef in a restaurant. The waiter is walking to the kitchen with an order from a regular customer who eats a steak every single Tuesday. The chef sees the customer sit down and immediately starts cooking the steak before the waiter even speaks. If the customer does order the steak, it is served instantly. The chef's guess paid off perfectly. If the customer suddenly orders fish, the chef throws the half-cooked steak in the trash and starts cooking the fish. Throwing the steak away is wasteful, but the chef cooks so fast that the restaurant still serves far more meals overall by guessing.

In our order pipeline, speculation is the CPU actually executing the shipping logic before the branch is confirmed:

if (item.inStock) {
    // (1) The CPU speculatively executes this heavy work
    let cost = calculateShippingTariff(item.weight);
    chargeAccount(user, cost);
}

At // (1), the CPU's ALU begins computing the tariff math and pulling the user's data into its registers. If item.inStock turns out to be true, the math is already finished and ready. If the item is actually out of stock, the CPU throws away the calculated cost and clears the modified registers as if they never happened. The program remains completely correct, but because the CPU gambled and did the work early, the whole pipeline runs dramatically faster.

Doing several things at once, and not in the order you wrote them. A pipeline overlaps the stages of consecutive instructions. Modern cores go two steps further, and both steps matter for understanding why real code runs at the speed it does.

The first step is being superscalar: a core has several of each kind of worker. Where the textbook CPU has one ALU, a modern core has four or more, plus several units for loading from memory and several for storing to it, and its front end can decode four to six instructions in every cycle. So in a single tick of the clock, several independent instructions can each be handed to a different worker and finish together.

The second step is out-of-order execution, and it is the one that surprises people: the CPU does not run your instructions in the order you wrote them. It reads ahead into a window of a few hundred upcoming instructions, works out which ones depend on which, and starts each one the moment the values it needs are ready. Look at three lines from our order pipeline:

const price = order.total;          // (1) may need a slow trip to main memory
const tax = price * 0.2;            // (2) needs (1)
const label = `#${order.id}`;       // (3) needs nothing from (1) or (2)

If order.total is not in the cache, line (1) waits around a hundred nanoseconds for main memory (1.6), which is hundreds of cycles. Line (2) cannot start until it arrives. But line (3) does not need either of them, so the CPU runs it during the wait instead of sitting idle. The results are held in a structure called the reorder buffer and only made official, in the original program order, once every earlier instruction has finished. That last part is what keeps the illusion intact: from the outside, the program always appears to have run line by line, exactly as written. Speculation, from the paragraphs above, is the same machinery running ahead past a branch, and Spectre (section 5) is what happens when the work that is thrown away still leaves a trace.

Why you should care as a programmer: a chain of instructions where each needs the one before cannot be overlapped, while independent work can. That is why a loop that follows a linked list (each step waits for the previous pointer to arrive from memory) is far slower than a loop over an array doing the same amount of arithmetic, even though both look like "one step per item".

5. The expert lens: when speculation broke security (Spectre)

For decades, "if wrong, discard the speculative results and no harm done" was accepted as obviously true. In 2018 it was shown to be false, in one of the most important security findings in the field's history — Spectre and Meltdown.

The flaw: when the CPU discards mis-speculated work, it correctly rolls back the registers and memory — but it does not roll back the cache (1.6). Data the speculative path touched is left sitting in the fast cache. An attacker can trick the predictor into speculatively reading a secret it isn't allowed to (a password, another program's memory), and although the CPU discards the read, the secret has now influenced which memory is cached. The attacker then measures memory access timing — cached data returns faster — and reads the secret back out through that side channel, one bit at a time. The security boundary was never crossed architecturally; it was crossed in timing. In simple terms, a side channel is like figuring out what someone is typing by listening to the sound of their keystrokes rather than looking at their screen. You are not breaking the official rules (the architecture) by looking at forbidden data directly. Instead, you are looking at an accidental physical side effect. Here, that side effect is time. When the CPU undoes its bad guess, it leaves the secret data sitting in the fast cache. The attacker then asks the CPU to do something that relies on that same data. If the CPU finishes almost instantly, the attacker knows the data was in the fast cache. By timing how long the CPU takes to answer, the attacker can slowly figure out the secret, one piece at a time, without ever formally asking for permission.

This broke the fundamental isolation assumption of shared computers — one program reading another's secrets, across the walls the OS enforces — and forced patches into virtually every CPU, operating system, and cloud on Earth, some costing real performance. The lesson is profound and worth carrying into every system you design: an abstraction can be perfectly correct in its stated model and still leak through a channel the model ignored — here, time. Performance optimizations can have security consequences invisible at the level you're reasoning at. You'll meet side channels again in cryptography (Part 8); their birthplace is right here, in the CPU's hunger for speed.

Next chapter: the trace above quietly assumed the CPU could get 5 and 7 instantly. It can't — memory is far slower than the CPU, and the gap is the single biggest factor in real-world performance. 1.6 builds the cache hierarchy that hides it, and shows why "cache-friendly" code can run 10× faster with identical logic.

Recall

• Von Neumann architecture: program and data share one memory; instructions are just numbers the CPU reads in sequence. The ALU, registers, and control unit are the adder, flip-flops, and FSM of 1.2.
• The CPU repeats fetch → decode → execute forever; the program counter tracks the next instruction. It understands only machine code from its ISA (x86-64, ARM, RISC-V); assembly is its readable form.
• The flagship trace: a + b → add eax, esi → bytes 01 F0 → the ALU's gates settling → transistors switching → voltages. Intent to electrons, unbroken.
• Speed comes from pipelining (overlap instructions), branch prediction (guess the next path >95% right), and speculative execution (run ahead on the guess).
• Spectre/Meltdown: speculation rolls back registers but not the cache, leaking secrets through a timing side channel — a correct abstraction breached through a channel (time) its model ignored.

Self-test: What made the stored-program idea revolutionary? Name the three cycle steps and what the PC does. Trace add eax, esi through fetch-decode-execute. Why does branch misprediction cost performance? In one sentence, how does Spectre steal a secret it never "officially" read?

Quiz Bank

FoundationalWhat is the Von Neumann architecture and why did it matter?

A design where a single memory stores both instructions and data, and the CPU reads instructions from it sequentially. It mattered because it made computers general-purpose and reprogrammable by loading new numbers instead of physical rewiring — instructions became data. Its components map directly to earlier chapters: ALU (combinational adder), registers (flip-flops), control unit (finite state machine).

FoundationalDescribe the fetch-decode-execute cycle.

Fetch: read the instruction at the program counter's address into the instruction register, and advance the PC. Decode: the control unit interprets the bits — operation, operands, addressing. Execute: carry it out (ALU operation, memory load/store, or a jump that rewrites the PC). Repeat every cycle. Control flow is just instructions that conditionally or unconditionally change the PC.

AppliedTrace what happens when the CPU runs `add eax, esi`.

Fetch: the bytes (01 F0) at the PC are loaded into the IR; PC advances. Decode: the FSM reads them as "ADD, source esi, destination eax" and routes those registers into the ALU. Execute: the ALU — the adder built from XOR/AND gates (1.2) of CMOS transistors (1.1) — computes the sum, and on the clock edge it's latched into eax. The abstract a + b becomes gates settling and voltages held in flip-flops.

AppliedWhat is an ISA, and name the three that matter today.

The instruction set architecture is the contract between software and hardware: the exact vocabulary of machine instructions a CPU understands, their encodings, and register model. Code compiled for one ISA won't run on another. The three current ones: x86-64 (Intel/AMD; PCs, servers), ARM (phones, Apple Silicon, increasingly servers — power-efficient), and RISC-V (open, royalty-free, rising).

InterviewWhat is pipelining and what limits its benefit?

Pipelining overlaps instruction stages (fetch/decode/execute/…) so multiple instructions are in flight at once, raising throughput toward one instruction per cycle without speeding up any single one. Limits: hazards — data hazards (an instruction needs a result not yet ready), and control hazards (branches, whose direction isn't known when the next fetch must happen). Branches are the big one, which is why CPUs add branch prediction and speculation; a misprediction flushes the pipeline, wasting the partially-done work.

InterviewWhy does branch misprediction hurt, and how do CPUs reduce it?

A pipelined CPU must fetch instructions after a branch before the branch resolves, so it guesses the direction and starts executing speculatively. A wrong guess means all that speculative work is discarded (a pipeline flush), and the pipeline refills from the correct path — costing many cycles. CPUs reduce it with a branch predictor that learns each branch's history and achieves >95% accuracy; well-predicted branches are nearly free, while unpredictable ones (e.g. data-dependent, random) are costly — which is why sorted data can make identical code run faster.

StaffExplain Spectre to a fellow engineer and state its general lesson for system design.

Modern CPUs speculatively execute past branches on predicted paths, discarding results if wrong. Spectre exploits that the discard rolls back registers/memory but not the cache: an attacker mistrains the branch predictor so the CPU speculatively reads a forbidden secret and uses it to index memory, leaving a footprint in the cache. Even though the read is "discarded," the attacker recovers the secret by timing memory accesses (cached = faster) — a side channel. General lesson: correctness in an abstraction's stated model doesn't guarantee security, because real implementations leak through channels the model ignores (here, time). Performance optimizations can silently create security boundaries you weren't reasoning about — a mindset you carry into caching, crypto, and multi-tenant systems.

Flashcards

FlashStored-program (Von Neumann) idea

Instructions and data live in the same memory; code is just numbers the CPU reads in order — reprogram by changing memory, not wiring.

FlashThe three cycle steps

Fetch → Decode → Execute, repeated every clock cycle.

FlashProgram counter (PC)

Register holding the address of the next instruction; jumps/branches/loops work by changing it.

FlashISA — three that matter

x86-64 (Intel/AMD), ARM (mobile/Apple), RISC-V (open).

FlashPipelining vs speculation

Pipelining overlaps instruction stages for throughput; speculation runs ahead on a predicted branch before knowing it's correct.

FlashSpectre in one line

Speculative execution leaves secrets in the cache; timing side-channel reads them out despite the speculation being "discarded."

Scenario Drill

DrillTwo versions of a loop do identical work over an array of numbers, but one runs several times faster — the only difference is that the fast one's data is sorted first. Using this chapter, explain why.

The loop almost certainly contains a data-dependent branch (e.g. if (value > threshold) …). On unsorted data the branch outcome is effectively random, so the branch predictor is wrong about half the time; each misprediction flushes the pipeline and wastes many cycles. On sorted data the branch is false for a long run and then true for a long run — highly predictable — so the predictor is right nearly always and speculation pays off, keeping the pipeline full. Identical logic, but the CPU's ability to guess the next instruction changed completely. (This is the famous "why is processing a sorted array faster" result.) The engineering takeaways: unpredictable branches are a real cost, and sometimes branchless code — computing both sides and selecting — beats a mispredicted branch.