3.6.9 — V8 Internals: How JavaScript Got Fast

Everything in this folder so far — closures, prototypes, the event loop — describes what JavaScript means. This page is about how it got fast: fast enough that a language designed in ten days now runs trading dashboards, IDEs, and half the world's servers. V8 is Google's JavaScript engine (Chrome, Edge, Node.js, Deno), and its core problem is brutal: JavaScript is dynamically typed with mutable everything (3.3), so almost nothing a compiler wants to assume is guaranteed. V8's answer, from 3.2: speculate — bet on what your code probably does, compile for that bet, and keep an exit if it goes wrong. Every mechanism on this page is that idea in a different costume, and each one ends in advice you can act on in a hot path.

1. The tiered pipeline: from source to speed

V8 parses source to an AST (3.1), then runs it through a ladder of tiers, each slower to compile but faster to run — paying compile cost only for code that proves it deserves it:

1. Ignition — compiles the AST to compact bytecode and interprets it. Cheap and immediate: most code runs once ever, and Ignition is how it runs. While interpreting, it collects type feedback — which shapes and types each operation actually sees.
2. Sparkplug — a baseline compiler: translates bytecode to unoptimized machine code almost instantly (no optimization analysis), removing pure interpretation overhead for warm code.
3. Maglev — the mid-tier optimizer: uses the type feedback for solid, quickly-generated optimized code for warm-but-not-scorching functions.
4. TurboFan — the heavyweight: for genuinely hot functions, performs aggressive speculative optimization (inlining, escape analysis, redundancy elimination) and emits machine code comparable to a static compiler's — guarded by assumptions from the feedback.

The safety valve is deoptimization: every TurboFan assumption carries a guard, and when a guard fails — the "always a number" parameter receives a string — execution bails out mid-function back to bytecode, the speculation is discarded, and the function may be re-optimized later with wider assumptions. Hence the practical smell: code that is usually consistent but occasionally weird can be worse than plainly slow code, because it pays optimize → deopt → reoptimize churn. (Node exposes this: node --trace-deopt, or DevTools performance profiles marking "deoptimized" frames.)

function add(a, b) { return a + b; }
add(1, 2);        // feedback: numbers — Maglev/TurboFan compile a pure integer add
add(3, 4);        // fast path: two machine instructions, no type checks in the loop
add("a", "b");    // guard fails → DEOPT → back to bytecode, speculation rebuilt

2. Hidden classes: giving shapeless objects a shape

The deepest dynamic-language problem: in C, user.name compiles to "load the word at offset 8" — one instruction — because the struct layout is known (1.5). A JavaScript object is a mutable bag of properties, so the naive implementation is a dictionary lookup per access. V8's fix: secretly reintroduce structs.

Every object gets a hidden class (V8 calls it a map; other engines say shape) describing its layout: which properties, in which order, at which offsets. Objects built the same way share one hidden class, and adding a property moves an object along a transition chain to the next class:

const p1 = {};          // hidden class C0 (empty)
p1.x = 1;               // → C1 {x @ offset 0}
p1.y = 2;               // → C2 {x @ 0, y @ 1}

const p2 = { x: 5, y: 9 };   // same properties, same ORDER → also C2 ✅ shared

const p3 = {};          // C0
p3.y = 2;               // → C1' {y} — DIFFERENT branch of the transition tree
p3.x = 1;               // → C2' {y, x} ≠ C2  // same keys, different order, different class

Figure 1 — The hidden-class transition tree. Objects that gain the same properties in the same order travel the same path and share a class (green) — V8 can then use fixed offsets, struct-style. The same keys added in a different order take a different path (red): different class, no sharing, optimizations lost.

Actionable consequences, straight from the mechanism: initialize every property in the constructor, unconditionally, in one fixed order (conditional/late additions fork the tree); never delete in hot code (it can knock the object into slow dictionary mode — set null instead); create objects with literals or classes rather than sprinkling properties across code paths.

3. Inline caches: remembering the answer at each call site

Hidden classes make layouts knowable; inline caches (ICs) make lookups free. The insight: a given line, like user.name inside a loop, almost always sees objects of the same hidden class. So V8 caches, at that call site: "for class C2, name is at offset 1." Next execution: one shape check, then a direct load — near-static-language speed. IC states escalate:

• monomorphic — the site has only ever seen one shape: fastest path;
• polymorphic — 2–4 shapes: small if-chain over cached entries, still good;
• megamorphic — many shapes: the cache gives up; generic (hash-lookup) access forever.

This is the mechanism behind the profiling advice "keep hot functions monomorphic": a utility invoked with ten differently-shaped configs isn't abstractly untidy — it concretely downgrades every property access inside it. ICs also feed the tiers: their recorded shapes/types are the feedback Maglev and TurboFan speculate on (section 1) — the three mechanisms are one system.

4. Numbers and array storage: representation matters

SMI vs HeapNumber. All JavaScript numbers are IEEE-754 doubles by spec (1.4), but V8 stores integers that fit (31-bit on 64-bit platforms, via pointer tagging) as SMI — SMall Integer — packed directly into the value word: no heap allocation, no pointer chase. Everything else becomes a heap-allocated HeapNumber. Loop counters and array indices staying in SMI range is a real, measurable win; a hot array that suddenly stores 3.5 upgrades its whole storage to doubles.

Elements kinds. Arrays carry a storage tag that only ever narrows in one direction: PACKED_SMI_ELEMENTS (dense ints — fastest) → PACKED_DOUBLE (a float appeared) → PACKED_ELEMENTS (any object) → HOLEY_* (a hole appeared — e.g. delete arr[3], sparse writes past the end, or new Array(1000) before filling). Transitions are one-way per array: go holey once, stay holey. Holey arrays pay extra checks on every access (the hole might mean "look up the prototype chain" — 3.6.4). Practical: keep arrays dense and type-homogeneous in hot code; build with push, not out-of-bounds writes; never delete an element (use splice or overwrite).

5. Orinoco: garbage collection without the pauses

V8's collector Orinoco applies 3.4's ideas tuned for interactive latency, resting on the generational hypothesis: most objects die young (temporaries, intermediate results), so treat young and old memory differently.

• Young generation ("nursery", small — a few MB): collected frequently by the Scavenger, a parallel copying collector — live survivors are copied out, the whole space is then free in one stroke; cost proportional to survivors, which are few. Objects surviving two scavenges get promoted to the old generation.
• Old generation: collected rarely by mark–sweep–compact. This is where the pause danger lives, so Orinoco attacks it three ways: incremental marking (interleaved in small slices with your code), concurrent marking/sweeping (on background threads while JavaScript runs), and parallel phases (all cores during the brief stop-the-world moments). Result: pauses measured in single-digit milliseconds — under a 16 ms frame budget — instead of the multi-hundred-ms freezes of naive mark-sweep.

Where closures meet the collector: a long-lived closure's context object (3.6.2) gets promoted to old space, so "small" leaked captures end up in the expensive-to-collect generation — one more reason leak hygiene matters. Allocation-heavy hot loops (fresh objects/arrays per iteration) show up as scavenger churn in profiles; reusing buffers or hoisting allocations out of loops is the classic fix.

6. The expert lens

Engine optimizations reward predictable code — write for the JIT. Hidden classes, ICs, and tier speculation all bet that your code sees consistent shapes and types. That converts vague style advice into mechanism: initialize all properties in the constructor in one order (stable hidden class); no delete, no late additions (transitions, dictionary mode); homogeneous, dense arrays (elements kinds); consistent argument shapes into hot functions (monomorphic ICs); consistent types through hot arithmetic (no deopt churn). None of it matters in cold code — premature micro-optimization is still a mistake — but in a genuinely hot path, shape consistency often beats algorithmic micro-tuning. The principle transfers to every JIT runtime (3.2): speculation rewards predictability; unpredictable code silently forfeits it.

Measure before believing — the engine may already have won. V8's tiers exist precisely so that idiomatic code is fast without help; folk "optimizations" (manual loop unrolling, avoiding closures dogmatically, micro-caching arr.length) are frequently neutral or harmful on a modern engine. The professional loop: profile (DevTools / node --prof / clinic.js), find the actual hot function, check its IC states and deopt log, fix the shape or algorithm, re-measure (Chapter 14.5 is the full discipline).

The same story runs everywhere. Hidden classes date to Smalltalk research (Self, 1980s); Python's dict versioning, Ruby's shapes, and the JVM's profile-guided inlining are siblings. Once you can narrate speculate → guard → deopt and shape → cache → megamorphic, you can reason about performance in any dynamic runtime you'll ever meet — the vocabulary is portable even when the engine isn't.

Next: with JavaScript's semantics and engine both open, 3.7.1 begins TypeScript — the static layer retrofitted on top.

Recall

• V8 pipeline: AST → bytecode on Ignition (interpreter + type feedback) → Sparkplug (baseline machine code) → Maglev (mid-tier) → TurboFan (aggressive speculative optimization) — with deoptimization bailing back to bytecode when a speculation guard fails.
• Hidden classes (maps/shapes) give dynamic objects struct-like layouts; same properties in the same order ⇒ shared class via the transition tree. Initialize everything in the constructor, one order; never delete in hot code.
• Inline caches remember "this shape → this offset" per call site: monomorphic (1 shape, fastest) → polymorphic (2–4) → megamorphic (many; generic slow path). IC feedback is what the optimizing tiers speculate on.
• SMI packs small integers into the value word (no heap allocation); other numbers are boxed HeapNumbers. Array elements kinds narrow one-way: packed-SMI → double → generic → holey (from delete/sparse writes) — keep arrays dense and homogeneous.
• Orinoco GC: generational — young space scavenged by parallel copying (cost ∝ survivors), old space mark–sweep–compact made incremental, concurrent, and parallel for single-digit-ms pauses. Long-lived closures promote to old space; allocation-heavy loops churn the scavenger.

Self-test: Name the four tiers and what each trades. Why do {x,y} and {y,x} not share a hidden class, and what should a constructor therefore do? Walk an IC from monomorphic to megamorphic and name the coding pattern that causes it. What makes an array "holey" and why does it stay that way? How does Orinoco keep pauses under a frame budget?

Quiz Bank

FoundationalDescribe V8's compilation pipeline and the role of deoptimization.

Source is parsed to an AST, then climbs tiers: Ignition compiles to compact bytecode and interprets it (instant startup; gathers type feedback); Sparkplug turns bytecode into unoptimized machine code for warm functions (kills interpretation overhead, no analysis); Maglev produces quickly-generated optimized code from feedback; TurboFan aggressively optimizes genuinely hot functions — inlining, escape analysis — emitting near-static-quality machine code guarded by speculation ("this parameter is always a SMI"). Deoptimization is the safety valve: when a guard observes a violation, execution bails out of the optimized code back to bytecode mid-function, and the function is later re-optimized with wider assumptions. The design point: pay compile cost only where runtime proves it's worth it, and make aggressive bets safe by keeping an exit.

FoundationalWhat are hidden classes and why does property order matter?

A hidden class (V8: map; generic term: shape) is a behind-the-scenes layout descriptor — which properties, what order, what offsets — that lets V8 treat dynamic objects like C structs: user.name becomes "load offset 1" instead of a dictionary lookup. Objects built identically share a class: each property addition transitions the object along a transition tree, so {x, then y} and {x, then y} land on the same class, while {y, then x} takes a different branch — same keys, different class, no sharing, optimizations lost. delete can demote an object to dictionary mode entirely. Hence the rules: initialize all properties in the constructor, unconditionally, in one fixed order; prefer literals/classes over scattered assignment; null-out instead of delete.

AppliedExplain inline caches and the monomorphic/polymorphic/megamorphic ladder.

An inline cache lives at each property-access site and remembers "for hidden class C, this property is at offset k." Because a given line usually sees the same shape repeatedly, later executions do one shape-check + direct load — near-native speed. States: monomorphic (one shape ever seen — fastest), polymorphic (2–4 shapes — small dispatch chain), megamorphic (many shapes — the cache abandons ship and every access takes the generic slow path, permanently for that site). Cause of megamorphism: one function fed many differently-shaped objects — ten config variants, heterogeneous rows, "options bag" APIs with wildly varying keys. The fix is shape discipline: normalize inputs to one standard shape at the boundary before they reach hot code. ICs double as the feedback source the optimizing tiers speculate on, so shape chaos also degrades tier-up quality.

InterviewWhat are SMIs and elements kinds, and what array habits follow from them?

SMI (small integer): V8 packs integers fitting ~31 bits directly into the tagged value word — no heap allocation, no dereference; other numbers are heap-boxed HeapNumbers (spec-wise all numbers are IEEE-754 doubles; SMI is a representation optimization). Arrays additionally carry an elements kind that only narrows: PACKED_SMI (dense integers, fastest) → PACKED_DOUBLE (one float appeared) → PACKED (objects/mixed) → HOLEY_* once a hole exists — from delete arr[i], writing past the end, or new Array(n) left unfilled. Transitions are one-way per array; holey access must also consider the prototype chain, adding checks forever after. Habits: keep arrays dense (build with push; no out-of-bounds writes; splice/overwrite instead of delete) and homogeneous (don't mix ints, floats, and objects in hot arrays); pre-size with new Array(n) only if you fill it immediately and completely.

InterviewHow does Orinoco keep GC pauses small? Walk through the young and old generations.

Orinoco is generational, exploiting "most objects die young." The young generation (a small nursery) is collected often by the Scavenger, a parallel copying collector: live objects are evacuated to the other half/promoted, and the whole space is reclaimed wholesale — cost proportional to survivors, which are few, so scavenges are sub-millisecond. Two-time survivors are promoted to the old generation, collected rarely by mark–sweep–compact — the dangerous pauses — which Orinoco splinters three ways: incremental marking (small slices interleaved with JavaScript), concurrent marking/sweeping on background threads while your code runs, and parallel use of all cores in the brief remaining stop-the-world windows. Net effect: pauses in single-digit milliseconds, fitting inside a 16 ms frame (3.4's stop-the-world problem, engineered away). Performance corollaries: allocation-heavy hot loops = scavenger churn (hoist/reuse allocations); long-lived closures/caches promote to old space where reclamation is costliest — leak hygiene (3.6.2) has a GC price tag too.

StaffA Node JSON-transform service shows p99 latency spikes. Profiles show one hot transform function, --trace-deopt logs repeated deopt/reopt cycles in it, and GC traces show heavy scavenger activity. Diagnose and fix, mechanism by mechanism.

Three signatures, three mechanisms. (1) Deopt churn: the hot function is being optimized on speculation, then hit with input violating it — classic causes: a field usually SMI but occasionally float/string/undefined (per-tenant data variance), or occasionally-missing keys changing shapes. Each cycle pays optimize + bailout + re-optimize; at p99 that's your spike. Find the exact guard in the --trace-deopt output ("wrong map", "not a Smi").

(2) Likely megamorphic ICs: if the service transforms many customer schemas through one function, its access sites see many hidden classes — permanent slow path. Check with --trace-ic or profile annotations. (3) Scavenger pressure: per-record allocation of intermediate objects/arrays floods the nursery; frequent scavenges add steady overhead and jitter.

Fixes, ordered: normalize input at the boundary — parse each record into one standard shape (constructor initializing every field, fixed order, explicit null for absent, consistent numeric types) so the transform sees a single hidden class → ICs go monomorphic, speculation holds, deopts stop; split genuinely different record families into separate specialized transform functions rather than one polymorphic funnel (per-site monomorphism beats one clever generic); cut allocations in the hot loop — reuse scratch objects/arrays, avoid intermediate .map().filter() chains in favor of one pass (or a lazy pipeline, 3.6.6) — calming the scavenger; then re-measure p99 and keep --trace-deopt clean in CI perf tests. Staff framing: the JIT is a bet on consistency; production variance is what breaks the bet — so enforce consistency at the data boundary, not deep in the hot path.

Flashcards

FlashV8 tiers

Ignition (bytecode + feedback) → Sparkplug (baseline) → Maglev (mid) → TurboFan (speculative, guarded) → deopt back to bytecode on violated guards.

FlashHidden class rule

Same properties, same order ⇒ shared class + fixed offsets. Constructor: initialize everything, one order; never delete (dictionary mode).

FlashIC ladder

Monomorphic (1 shape, fastest) → polymorphic (2–4) → megamorphic (many; permanent generic path). Fix: standard shapes at the boundary.

FlashSMI / HeapNumber

Small ints packed in the value word — free; other numbers heap-boxed. Keep counters/indices integral in hot loops.

FlashElements kinds

Packed-SMI → double → generic → holey; one-way. delete/sparse writes make holes; holey pays prototype checks forever. Keep arrays dense + homogeneous.

FlashOrinoco in one line

Generational: parallel copying Scavenger for the nursery (cost ∝ survivors); incremental + concurrent + parallel mark-sweep-compact for old space → single-digit-ms pauses.

Scenario Drill

DrillA dashboard's chart-rendering loop (60 fps target) stutters. DevTools shows the frame function itself is fast when sampled, but frames periodically blow past 16 ms with 'Minor GC' and occasional 'Recompile/Deoptimize' entries. The code builds a fresh {x, y, label, meta} point object per datum per frame, where meta is sometimes omitted and y is sometimes null. Fix it with this page's mechanisms.

The clues map exactly. Minor GC entries: ~thousands of fresh point objects per frame flood the young generation; the Scavenger runs mid-frame and, though each scavenge is fast, several per frame plus copying survivors bursts the 16 ms budget — allocation rate, not code speed, is the killer (why sampling says "fast").

Deopt entries: meta sometimes-omitted means points travel two hidden-class paths ({x,y,label} vs {x,y,label,meta}), and y: null sometimes poisons the numeric speculation — the render function tiers up on one shape/type, then deopts when the other arrives, recompiling mid-animation.

Fixes: (1) One standard shape — construct every point with all four fields, meta: null when absent, y always a number (use NaN sentinel rather than null if "missing" is meaningful) → single hidden class, monomorphic ICs in the loop, speculation holds. (2)

Stop allocating per frame — reuse a pre-allocated pool of point objects (or better, restructure hot data as typed arrays / parallel arrays: xs: Float64Array, ys: Float64Array, with labels/meta in side arrays touched only on interaction) → allocation in the frame loop drops to ~zero, scavenges disappear from the frame path; typed arrays also guarantee packed, homogeneous storage (section 4) with no elements-kind surprises. (3) Hoist any remaining temporaries (scratch vectors, format buffers) out of the loop. (4) Verify: Performance panel shows GC bars gone from frames and no recompile entries; --trace-deopt clean under a soak. Principle to state: at 60 fps the enemy is per-frame allocation and shape variance; hot loops want pooled, standard, homogeneous data — the same discipline games and visualization engines are built on.