Appearance
3.2 — Compile, Interpret & JIT
3.1 followed source code through a compiler and out the other side as an executable file — a finished, self-contained program the OS can load and run (2.2). That model describes C, C++, Rust, and Go perfectly. But it plainly does not describe how you use JavaScript in a browser, or Python at a prompt: there, you type code and it runs immediately — no build step, no .exe produced. Something different is happening.
This chapter maps the full spectrum of "how does source code actually get executed," and it turns out to be one of the most consequential design decisions in any language. At one end sits ahead-of-time compilation (translate everything first, run later); at the other, interpretation (read and execute the source as you go). In between lies the bytecode virtual machine, and layered on top of that, the technique that makes modern JavaScript run at speeds its designers never imagined: the JIT compiler. Understanding this spectrum explains why Java servers are slow for their first minute, why Python is "slow," why a browser can run a demanding game, and why serverless functions have a cold-start problem.
1. Two strategies, and the trade-off between them
There are two fundamental ways to get from source code to execution, and the classic analogy is translation between human languages.
Ahead-of-time (AOT) compilation is like hiring a translator to convert an entire book into another language, printing it, and handing you the finished translation. All the work happens before anyone reads. This is 3.1's pipeline: the compiler transforms the whole program into native machine code (1.5) for a specific CPU, producing an executable.
Interpretation is like a live interpreter standing beside you at a foreign meeting, translating each sentence as it is spoken. There's no finished translated document — the work happens continuously, during the conversation. An interpreter is a program that reads your source (usually after parsing it to an AST, 3.1) and directly performs what each construct says, one node at a time, without ever producing machine code.
Their trade-offs are close to exact opposites, and every entry follows from "when does the translation work happen?":
| AOT compiled | Interpreted | |
|---|---|---|
| Translation happens | once, before running | continuously, while running |
| Execution speed | fast (native code, heavily optimised) | slow (re-analysing the same code every time) |
| Startup | instant (already translated) | instant (no build step) |
| Development loop | slow (must rebuild to test a change) | fast (edit, run immediately) |
| Portability | the binary works on one CPU+OS | the source runs anywhere the interpreter exists |
| Errors found | many caught at compile time | mostly at runtime, when that line executes |
The speed gap is the crucial part, and it's worth seeing why it's so large. Consider a loop running a million times. The compiler analyses total = total + price once, decides total lives in a register, and emits a couple of machine instructions (1.5) that execute a million times. The interpreter re-walks that same AST node a million times, and on every single iteration asks: what kind of node is this? An addition. What are its operands? Look up total in a table of variables. What type is it — a number, a string? Dispatch to the right addition routine. Store the result back into the table. That's dozens of operations of bookkeeping per iteration to accomplish one addition — commonly 10–100× slower than compiled code. The interpreter isn't slow because interpretation is inefficient; it's slow because it re-discovers the same facts on every pass. Hold that sentence — the entire rest of this chapter is about clawing that waste back.
2. The middle ground: bytecode and virtual machines
Pure interpretation of an AST is wasteful in an obvious way: walking a tree of objects, chasing pointers (1.6) between nodes, is a poor use of a CPU that would rather stream through a flat array of simple instructions. So most "interpreted" languages don't actually interpret the AST. They add a step.
The source is compiled — ahead of time, quickly — not into machine code, but into bytecode: a compact, flat sequence of simple instructions for an imaginary, idealised computer. Then a program called a virtual machine (VM) executes that bytecode. The VM is essentially a software CPU: it holds an instruction pointer, fetches the next bytecode instruction, decodes it, and performs it — the fetch-decode-execute cycle of 1.5, implemented in software rather than silicon.
This buys two things at once. Speed over AST-walking: bytecode is flat, compact, and pre-digested — the parsing and much of the name resolution already happened, so the VM's per-instruction work is far smaller (though still well short of native code). Portability: the bytecode targets the imaginary machine, so the same bytecode file runs on any platform that has a VM — this is the literal meaning of Java's slogan "write once, run anywhere," and you'll notice it's exactly 3.1's intermediate-representation idea, except the IR is shipped to users instead of consumed internally.
Nearly every language you'd call "interpreted" actually works this way: Java compiles to .class bytecode run by the JVM (Java Virtual Machine); C# compiles to IL run by the .NET CLR; Python compiles to bytecode (those .pyc files in the __pycache__ folder — that's simply Python caching the compiled bytecode so it needn't re-compile unchanged source next time) run by the CPython VM; JavaScript engines do the same internally. So "compiled vs interpreted" is rarely a clean binary — it's a spectrum, and most modern languages sit in the middle.
3. The JIT: compiling at the last possible moment
Bytecode VMs are faster than AST-walking, but still far from native speed — the VM is still re-decoding each instruction every time through a loop. The breakthrough that closes most of the remaining gap is the Just-In-Time compiler (JIT), and its central insight is beautiful:
Most programs spend most of their time in a small fraction of their code. So don't compile everything — watch the program run, find the parts that are actually hot, and compile only those to native machine code, while it runs.
The word "hot" is literal jargon: a hot path is code executed many times (a loop body, a frequently-called function). A JIT-enabled VM therefore starts by interpreting bytecode (fast to start, no compilation delay), while a profiler counts how often each function and loop runs. When a counter crosses a threshold, the VM stops and compiles that function to optimised native code, then patches things so future calls jump straight to the compiled version. This is tiered compilation: start cheap and interpreted, escalate the hot parts to expensively-optimised native code.
Here is the genuinely surprising part, and the reason JIT is not merely "compiling late": a JIT can produce code that an ahead-of-time compiler cannot, because it knows things only observable at runtime. An AOT compiler must generate code that is correct for every possible input and situation. A JIT watches the actual execution and can speculate on what it observes. If a function add(a, b) has been called ten thousand times and a and b were integers every single time, the JIT emits a fast native integer addition — skipping all the type-checking bookkeeping that made interpretation slow. That's a bet, not a proof, so it also inserts a cheap guard: a quick check that the assumption still holds. If someone eventually calls add("hello", "world"), the guard fails and the VM performs deoptimisation — discarding the specialised code and falling back to the interpreter (then possibly re-optimising with the new information). This speculate-and-guard cycle is how a dynamically-typed language like JavaScript reaches within a small factor of C, and it's the direct answer to section 1's complaint: the JIT stops re-discovering the same facts by baking them into compiled code, while keeping a safety net for when they change.
You'll meet this machinery concretely in Chapter 3.6 — V8 (the engine in Chrome and Node.js) uses an interpreter called Ignition plus an optimising JIT called TurboFan, exactly this design. The JVM does the same with its C1 (fast) and C2 (heavily optimising) compilers.
4. The costs: warmup, memory, and unpredictability
JIT compilation isn't free, and its costs explain real, everyday production behaviour.
Warmup. For the first seconds (or minutes) of a program's life, code is interpreted and the profiler is still gathering data — so the program runs at a fraction of its eventual speed, then gradually accelerates as hot paths get compiled. This is why a freshly-started Java or .NET server is noticeably slow and why serious benchmarks discard the first results. It's also why cold starts hurt in serverless platforms (Part 13): a function invoked once and torn down never gets to run long enough to be optimised, so it always pays interpreted speed and startup cost — a major reason serverless favours lightweight runtimes and why AOT options (GraalVM native images, .NET AOT) exist specifically to eliminate warmup.
Memory and CPU overhead. The VM must hold the interpreter, the profiler's counters, the compiler itself, and the generated native code — so a JIT runtime uses substantially more memory than an equivalent AOT binary, and spends some CPU compiling instead of doing your work. On a memory-capped container (2.8) this matters.
Unpredictability. Performance varies over time and can regress suddenly when a deoptimisation fires. For most software this is a fine trade, but for real-time systems (audio processing, trading, control systems) where a missed deadline is a failure, an unpredictable pause is unacceptable — which is why such systems favour AOT-compiled languages with no JIT and no garbage-collection pauses (Chapter 3.4).
So the full spectrum, and how to read a language's position on it:
| Approach | Examples | Startup | Peak speed | Portability |
|---|---|---|---|---|
| AOT → native | C, C++, Rust, Go | instant | highest, predictable | rebuild per platform |
| Bytecode + interpreter | CPython | fast | low | high |
| Bytecode + JIT | Java/JVM, C#/.NET, JavaScript/V8 | slow (warmup) | very high after warmup | high |
| Pure AST interpretation | shell scripts, simple DSLs | instant | lowest | high |
5. The expert lens
"Compiled vs interpreted" is a property of an implementation, not a language — and the distinction misleads beginners. Nothing about the language Python forbids compiling it to native code (PyPy JIT-compiles it; Cython compiles it AOT), and nothing about C forbids interpreting it (interpreters exist). What people usually mean by "Python is interpreted" is "the standard CPython implementation compiles to bytecode and interprets it, without a JIT." Getting this right matters because it reframes performance questions correctly: "Python is slow" is really "CPython's interpretation loop and its dynamic-typing bookkeeping are slow for tight numeric loops" — which immediately suggests the real fixes practitioners use: push hot loops into AOT-compiled native libraries (NumPy is C under a Python skin), use a JIT implementation (PyPy, Numba), or compile the hot part (Cython, Rust extension). The language isn't the bottleneck; a particular execution strategy for a particular workload is.
The whole spectrum is one trade: when do you pay for translation, and how much do you know when you pay? Pay early (AOT) and you get instant startup and predictable speed, but you must be conservative because you know nothing about the actual run. Pay late (JIT) and you can exploit what actually happened — real types, real branch outcomes, real hot paths — buying speed the AOT compiler couldn't justify, at the cost of warmup, memory, and predictability. Everything else follows: the cold-start problem, why AOT is returning for serverless and CLI tools, why long-running servers love JITs (they warm up once and then run hot for days), and why WebAssembly (a portable bytecode-like format designed to be compiled to native quickly) is attractive for both the browser and the edge. When you next choose a runtime, ask "is this workload long-lived enough to amortise warmup?" — that single question resolves most of the decision.
Bytecode is a security and tooling boundary too, not just a speed trick. Because bytecode is a well-defined, machine-neutral format, it can be inspected, verified, and sandboxed before execution — the JVM famously verifies bytecode for safety properties before running it, and browsers do the same for WebAssembly. This is why VM-based platforms are the natural home for running semi-trusted code, and it connects straight to the isolation spectrum of 2.9: a VM boundary in software (the language VM) is another rung on the same ladder as containers and hypervisors. It also means bytecode is decompilable — Java and C# binaries can be read back into near-source form far more easily than native machine code, which is why obfuscators exist for those ecosystems.
Next chapter: we've repeatedly said the interpreter must "check what type this value is" and the JIT must "guard that it's still an integer." That raises the question this whole discussion has been circling: what is a type, when is it checked, and what does a language gain or lose by insisting you declare them? Chapter 3.3 takes on type systems.
Recall
- AOT compilation translates the whole program to native machine code before running (fast, predictable, per-platform binaries). Interpretation executes the source/AST directly as it goes (instant edit-run loop, portable, but ~10–100× slower). The slowness is re-discovering the same facts on every pass, not interpretation per se.
- Most "interpreted" languages actually compile to bytecode — flat instructions for an imaginary machine — executed by a virtual machine (a software CPU running fetch-decode-execute). Bytecode gives portability ("write once, run anywhere") and beats AST-walking. Python's
__pycache__/.pycfiles are cached bytecode. - A JIT interprets first while a profiler finds hot paths, then compiles just those to native code (tiered compilation). It can beat AOT by speculating on runtime facts (e.g. "these are always integers"), protected by cheap guards; when a guard fails it deoptimises back to the interpreter.
- JIT costs: warmup (slow first seconds — hence serverless cold starts and discarding early benchmark runs), extra memory/CPU for the compiler and generated code, and unpredictable pauses (bad for real-time).
- "Compiled vs interpreted" describes an implementation, not a language (CPython vs PyPy vs Cython all run Python). The core trade: pay for translation early with little knowledge, or late with real runtime knowledge.
Self-test: Why is an interpreter typically 10–100× slower than compiled code — what is it repeating? What does a bytecode VM buy over walking an AST? How can a JIT produce faster code than an AOT compiler? What are guards and deoptimisation? Why do serverless platforms suffer from JIT warmup?
Quiz Bank
FoundationalWhat is the difference between a compiler and an interpreter?
A compiler translates the entire program ahead of time into another form (usually native machine code), producing an artifact that is run later — all translation work happens before execution. An interpreter reads the program (typically its AST or bytecode) and directly performs its instructions as it goes, with no separate translated output — translation work happens during execution. Consequences: compiled code runs much faster and catches many errors before running; interpreted code starts instantly with no build step, is portable to any machine with the interpreter, and surfaces most errors only when the line executes.
FoundationalWhy is interpreted code typically much slower than compiled code?
Because the interpreter re-does the same analysis on every execution. In a loop, the compiler analyses total = total + price once and emits a couple of machine instructions that then run a million times. The interpreter, on each of the million iterations, must re-examine the AST/bytecode node, look up each variable in a table, check the operand types, dispatch to the right operation, and store the result back — dozens of bookkeeping steps to accomplish one addition. That repeated re-discovery, not interpretation itself, is the 10–100× cost — and eliminating it is exactly what a JIT does.
AppliedWhat is bytecode and what does a virtual machine do with it?
Bytecode is a compact, flat sequence of simple instructions for an imaginary (idealised) computer, produced by compiling the source ahead of time. A virtual machine is a program that executes it — effectively a CPU in software, running the fetch-decode-execute cycle over bytecode instructions. Benefits: portability (the same bytecode runs anywhere a VM exists — Java's "write once, run anywhere") and speed over AST-walking (flat, pre-digested instructions are cheaper to execute than chasing pointers through a tree). Used by the JVM (.class), .NET (IL), and CPython (.pyc files cached in __pycache__).
AppliedWhat are .pyc files and the __pycache__ folder?
They're Python's cached bytecode. When you run a Python module, CPython first compiles the source to bytecode, then its VM executes that bytecode. To avoid re-compiling unchanged source on every run, it saves the compiled bytecode as a .pyc file inside a __pycache__ directory, and reuses it as long as the source hasn't changed (validated by timestamp/hash). So they're a build cache, not something you write or ship deliberately — deleting them only costs a one-time recompile. Their existence is direct evidence that "interpreted" Python is really compile-to-bytecode-then-interpret.
InterviewHow does a JIT compiler work, and how can it outperform an ahead-of-time compiler?
A JIT starts by interpreting bytecode (instant startup) while a profiler counts executions to find hot paths; when a function/loop crosses a threshold, it's compiled to optimised native code and future calls use that version (tiered compilation). It can beat AOT because it knows things only visible at runtime: an AOT compiler must emit code correct for every possible input, whereas a JIT observes that (say) a function has only ever received integers and speculatively emits specialised fast integer code, skipping dynamic type bookkeeping. Safety comes from a cheap guard that re-checks the assumption; if it ever fails, the VM deoptimises back to the interpreter and may re-optimise with new information. This speculate-and-guard cycle is how dynamically-typed JavaScript approaches native speed.
InterviewWhat are the downsides of JIT compilation?
Three. Warmup: the program runs interpreted and slow until the profiler identifies hot code and compiles it, so early execution is far below peak — which is why fresh JVM/.NET servers are sluggish at first and benchmarks discard initial runs. Resource overhead: the runtime carries an interpreter, profiler, compiler, and generated native code, using notably more memory (and some CPU compiling rather than doing your work) than an AOT binary — significant under container memory limits. Unpredictability: performance shifts over time and can regress abruptly when a deoptimisation fires, which is unacceptable for real-time systems with hard deadlines (audio, trading, control), so those prefer AOT with no JIT.
StaffYour team runs a Java service on a serverless platform and sees terrible latency on infrequently-invoked endpoints, but fine latency under steady load. Explain and give options.
This is JIT warmup meeting serverless cold starts. Under steady load the JVM stays alive, the profiler identifies hot paths, and the JIT compiles them to native code — so latency is good. For infrequently-invoked endpoints, the platform tears down idle instances, so each call hits a fresh JVM: it pays JVM startup plus runs fully interpreted (pre-warmup), never executing long enough to trigger optimisation — hence dramatically worse latency. The mismatch is structural: JIT amortises compilation cost over a long-lived process, and serverless functions are short-lived by design. Options, roughly in order: (1)
eliminate warmup with AOT — compile to a native image (GraalVM native image for Java, or .NET AOT), giving instant startup and predictable speed at the cost of some peak throughput and build complexity — usually the right answer for serverless; (2) keep instances warm — provisioned concurrency / scheduled pings so a warmed JVM is always available (costs money, partly defeats scale-to-zero); (3)
tune the runtime — reduce startup work (lazy initialisation, smaller dependency graph), use tiered-compilation settings that favour fast warmup, or class-data sharing; (4) change the deployment model — move steady-state endpoints to a long-running container (2.9) where the JIT pays off, and reserve serverless for genuinely bursty work; (5)
change runtime for those endpoints to one with negligible startup (Go, Rust — AOT). The staff framing: match the execution strategy to the process lifetime — JIT wins for long-lived processes, AOT wins for short-lived ones — and this question is the clearest real-world instance of that rule.
Flashcards
FlashAOT compilation vs interpretation
AOT: translate whole program to machine code before running (fast, per-platform). Interpretation: execute source/AST directly while running (portable, instant loop, 10–100× slower).
FlashWhy interpreters are slow
They re-analyse the same code on every pass — re-checking node type, looking up variables, dispatching by type — instead of deciding once like a compiler.
FlashBytecode + virtual machine
Source compiled to compact instructions for an imaginary machine; a VM (software CPU) executes them. Gives portability + speed over AST-walking. JVM, .NET CLR, CPython.
FlashWhat __pycache__/.pyc files are
Cached compiled bytecode so Python needn't recompile unchanged source each run.
FlashJIT in one sentence
Interpret first while profiling; compile the hot paths to optimised native code at runtime (tiered compilation).
FlashGuard and deoptimisation
A guard cheaply re-checks a speculative assumption (e.g. "still integers"); if it fails, the VM deoptimises — discards the specialised code and falls back to the interpreter.
FlashThree JIT costs
Warmup (slow start), memory/CPU overhead (interpreter+profiler+compiler+code), unpredictable performance (deopt pauses) — bad for real-time and serverless.
Scenario Drill
DrillA data scientist says 'Python is too slow for this numeric loop, we must rewrite the service in C++.' Using this chapter, challenge that conclusion and propose cheaper alternatives.
First, correct the framing: "Python is slow" is about an implementation and a workload, not the language. The cost is CPython's interpretation loop — per-iteration bookkeeping (decode the operation, look up names, check dynamic types, dispatch, box results) repeated millions of times, precisely the re-discovery problem of section 1. A tight numeric loop is the worst case for that model, but it's also the case with the best-known cheap fixes, so a full C++ rewrite (expensive, risky, loses the ecosystem and iteration speed) should be the last resort, not the first. Cheaper alternatives, in ascending effort: (1)
Vectorise with NumPy — express the loop as array operations; NumPy is AOT-compiled C underneath, so the loop runs in native code with the Python-level bookkeeping paid once rather than per element — frequently a 10–100× win for a few lines changed. (2) JIT the hot function — Numba compiles annotated numeric Python to native code at runtime (the JIT of section 3), or run on PyPy, a JIT implementation of Python; near-native speed with essentially no rewrite. (3)
AOT-compile the hot part — Cython or a small Rust/C extension for just the kernel, keeping the rest of the service in Python; this is the standard "hot loop in native code, glue in Python" architecture the whole scientific stack uses. (4) Only if the entire service is compute-bound and none of the above suffices, consider a rewrite. Also verify with a profiler that this loop truly dominates before optimising anything (1.5/Part 14) — and note the trade you'd be accepting with C++: better peak and predictable performance, at the cost of development speed, memory-safety risk, and ecosystem. The transferable lesson: when a language "is slow," identify which execution strategy is costing you and change that, rather than changing languages.