2.7 — Input/Output

Every chapter so far has quietly assumed that when a program needs data from a disk or network, the data just appears. 2.6 even said the file system "reads a block" as if that were instant. It is not. A CPU executes an instruction in a fraction of a nanosecond; a disk takes milliseconds, a network round-trip even longer (1.6 — recall that to a CPU, waiting for a disk is like you waiting a year). So the deepest question in I/O is not "how do we move the bytes" but "what does the blazingly-fast CPU do during the agonizing wait, and how does it find out the wait is over?" The answers to that one question — interrupts, DMA, and above all the distinction between blocking and non-blocking I/O — are the foundation of everything from why your OS stays responsive to why Node.js can handle 10,000 connections on a single thread. This chapter is, in a real sense, the missing prequel to the Node.js event loop you'll meet in Part 3.

1. Talking to devices: polling vs interrupts

First, the basic mechanics. A CPU communicates with a device (disk controller, network card, keyboard) through the device's registers and a driver (2.1). The CPU says "go read sector 91,442," and then the device takes milliseconds to physically do it. During those milliseconds, how does the CPU learn when the data is ready? Two strategies, and the difference is enormous:

• Polling (busy-waiting): the CPU repeatedly asks the device "done yet? done yet? done yet?" in a tight loop. Simple, but catastrophic — the CPU burns millions of cycles doing nothing but asking, cycles it could have spent running other programs. It's standing at the microwave asking "ready?" every second instead of doing something useful.
• Interrupts: the CPU issues the request and then moves on to other work entirely. When the device finishes, it raises an interrupt — an electrical signal that forces the CPU to stop whatever it's doing, jump to a special kernel function called an interrupt handler (or ISR, interrupt service routine), deal with the completed I/O (e.g. "the data has arrived, wake the process that wanted it"), and then resume what it was doing. It's setting the microwave's timer and going to do the dishes; the ding interrupts you only when it's actually done.

Every modern system is interrupt-driven, for the obvious reason: it lets the CPU stay productive during the vast dead time of I/O. This is the hardware mechanism underneath the 2.2 "Blocked" state — a process waiting on I/O is set aside (using no CPU), and the interrupt is what eventually moves it back to Ready. It's also, note, the very same mechanism as the timer interrupt that powers preemptive scheduling (2.3): interrupts are how the hardware grabs the CPU's attention for any asynchronous event, whether a device finishing or a time slice expiring.

2. DMA: don't make the CPU move the bytes

There's a second inefficiency to kill. Suppose a network card has received 64 KB of data. Someone must copy those 64 KB from the device into RAM. If the CPU did it — reading each word from the device register and writing it to memory — it would be tied up for the whole transfer, doing dumb copying instead of real work. So machines include a DMA (Direct Memory Access) controller: a small dedicated piece of hardware whose only job is moving data between devices and RAM, without the CPU. The flow becomes beautifully hands-off: the CPU tells the DMA controller "move the incoming data to this memory region," then goes back to running programs; the DMA controller shuffles the bytes on its own; and only when the whole transfer is complete does it raise a single interrupt to say "done." So the CPU is involved for the setup and the completion, but not the thousands of cycles of copying in between.

Figure 1 — DMA. The CPU sets up the transfer, then the DMA controller moves data device↔RAM on its own, freeing the CPU. A single interrupt signals completion. This is why large transfers don't monopolize the CPU.

Interrupts + DMA together are why a modern machine can download a file, play music, and run your code all at once without the CPU drowning: devices do their slow work independently and only interrupt the CPU at meaningful moments, and DMA spares the CPU the drudgery of copying. With that hardware foundation, we reach the question that actually shapes how software is written.

3. Blocking vs non-blocking: the choice that shapes servers

When your program calls read() on a socket that has no data yet, what happens? There are two fundamentally different behaviors, and choosing between them is one of the most consequential design decisions in all of systems programming.

• Blocking I/O (the default): the read() syscall does not return until data is available. Your thread goes to sleep (Blocked, 2.2), consuming no CPU, and the kernel wakes it (via an interrupt) when data arrives. Simple and intuitive — the code reads top-to-bottom as if I/O were instant — but with a hard limit: that thread can do nothing else while it waits. To handle another connection simultaneously, you need another thread.
• Non-blocking I/O: the read() syscall returns immediately, either with whatever data is ready or with a special "would block — nothing here yet" status (EAGAIN). Your thread is never put to sleep; it stays in control and can go do other things, checking back later. More complex to program, but it unlocks a superpower: one thread can juggle many I/O operations at once, none of them blocking it.

This choice creates two entire architectures for a server, and the tension between them is the crux:

Figure 2 — Two server architectures. Blocking: one thread per connection, each sleeping on its I/O — simple but thousands of threads cost memory and context-switching. Non-blocking: one thread watching many connections, working on whichever is ready — scales to huge connection counts.

The blocking, thread-per-connection model is simple but hits a wall famously called the C10k problem: to handle 10,000 simultaneous connections, you'd need ~10,000 threads, and from 2.3 you know the cost — each thread needs a stack (megabytes of memory), and the scheduler drowns in context switches among thousands of mostly-Blocked threads. Most of those threads are just waiting on slow network I/O, doing nothing, yet still costing resources. It doesn't scale. The non-blocking model sidesteps it entirely — but to make one thread watch thousands of connections efficiently, we need one more piece.

4. I/O multiplexing: select, poll, epoll

If one thread is to manage a thousand non-blocking sockets, it faces a question: which of these thousand connections has data ready right now? It can't just try each one in a loop (that's polling again, wasteful). It needs the kernel to tell it "of all these sockets you care about, here are the ones that are ready." That capability is I/O multiplexing, and its evolution is a story of scaling:

• select (and poll) — the original: hand the kernel a list of all your file descriptors and say "block until at least one is ready, then tell me which." It works, but has a fatal flaw at scale: on every single call, you pass the entire list of (say) 10,000 descriptors, and the kernel scans all of them to see which are ready — O(n) work every time, even if only one socket became ready. With thousands of connections, this linear scan on every event dominates, and the whole point (efficiency) is lost. select also has a hard limit (typically 1024 descriptors).
• epoll (Linux; kqueue on BSD/macOS; IOCP on Windows) — the modern solution, and the hero of the C10k story. Instead of re-scanning everything each call, epoll is stateful: you register your interest in each socket once (epoll_ctl), and the kernel maintains a ready list behind the scenes — as interrupts fire and sockets become ready, the kernel adds them to that list. When you call epoll_wait, it hands you only the sockets that are actually ready, in roughly O(1) with respect to the total number you're watching (you pay only for the active ones, not the idle thousands). This is the breakthrough that made a single thread efficiently manage tens of thousands of connections — and it is exactly the mechanism at the heart of Nginx, Redis, and Node.js.

This is the payoff the whole chapter has been building toward: an event loop. One thread sits in a loop calling epoll_wait, which sleeps efficiently (no CPU) until the kernel says "these sockets are ready"; the thread then does a tiny bit of work for each ready socket (read the available data, run your callback), and loops back. No thread-per-connection, no blocking, no wasted scanning — one thread, thousands of connections, driven by readiness events. When you learn the Node.js event loop (via the libuv library) in Part 3, know that this — non-blocking sockets plus epoll/kqueue — is the engine inside it. Node's "single-threaded, non-blocking I/O" is this OS machinery, wrapped in JavaScript.

5. Completion-based I/O: io_uring

There's one more frontier, because even epoll has a cost: it's readiness-based (it tells you a socket is ready, then you make a separate syscall to actually read the data — so every I/O still costs syscalls, and from 2.1 you know syscalls aren't free, especially post-Spectre). Linux's io_uring (2019) is a newer, completion-based model that attacks this. It sets up two ring buffers shared between the application and the kernel (in shared memory): a submission queue where the app posts I/O requests, and a completion queue where the kernel posts results. The app can submit thousands of operations and collect their results with almost no syscalls at all — it just writes to the shared submission ring and reads the completion ring. Instead of "tell me when it's ready so I can read it," io_uring is "here's a batch of full operations; put the results here when done." For I/O-heavy servers this dramatically cuts the syscall overhead that even epoll incurs, and it's steadily being adopted by high-performance systems (and, increasingly, by runtimes like Node under the hood).

6. Measuring I/O: IOPS

Finally, the metric you'll see in every cloud storage spec and database sizing discussion: IOPS (Input/Output Operations Per Second) — how many discrete read/write operations a storage device or system can perform per second. It matters because, as 2.6 showed, workloads made of many small operations are bottlenecked not by raw bandwidth (MB/s) but by the count of operations — a database doing thousands of tiny random reads lives and dies by IOPS, not throughput. A spinning disk offers maybe 100–200 IOPS (limited by physical seek time); an SSD offers tens of thousands to millions. Cloud providers sell IOPS as a provisioned resource (you literally pay for guaranteed IOPS on a volume), which is why understanding whether your workload is IOPS-bound (many small ops) or throughput-bound (few large ops) directly shapes both architecture and cost — a recurring theme when we reach cloud storage in Part 13.

7. The expert lens

This chapter is why the async/event-loop model exists — and when it wins. Return to the 2.3 distinction: I/O-bound work (waiting on network/disk) vs CPU-bound work (computing). For an I/O-bound server — thousands of connections mostly waiting — the thread-per-connection model wastes enormous resources on threads that do nothing but sleep, and drowns in context switches. The event-loop model (non-blocking I/O + epoll) handles them all on one thread, because while any connection waits, the thread simply works on others — perfectly matched to work that's mostly waiting. This is the entire architectural bet of Node.js, Nginx, and Redis, and now you can articulate why it wins: it eliminates per-connection thread cost for a workload where the threads would just be blocked anyway. The flip side, equally important: for CPU-bound work, the single event-loop thread is a liability — one heavy computation blocks the entire loop, freezing all connections (the infamous "don't block the event loop"), which is why such runtimes offload CPU work to worker threads or separate processes. Knowing which regime you're in is the whole judgment.

Zero-copy — the ultimate "cross the boundary less." Consider serving a file over the network the naive way: read() copies file data from the kernel's page cache into your program's buffer (kernel→user copy), then write() copies it from your buffer back into the kernel's socket buffer (user→kernel copy) — the data crosses the 2.1 boundary twice and is copied twice, though your program never even looked at it. Zero-copy techniques like sendfile() tell the kernel "send this file to this socket directly," so the data goes from page cache to network card without ever entering user space — no copies, no boundary crossings. This is how high-performance servers and CDNs (Part 13) serve static files at line rate, and it's the same principle as batching syscalls and DMA: the fastest data movement is the one that doesn't happen. When you see sendfile, memory-mapped I/O (2.5), or "zero-copy" in a system's design, this is the win being claimed.

Backpressure — the consequence of speed mismatches. I/O connects components running at wildly different speeds (a fast producer, a slow consumer, or vice versa). If a fast source pours data into a slow sink faster than it can be handled, unbounded buffering fills memory until the system falls over. The disciplined answer is backpressure: a feedback signal that makes the fast side slow down to the pace the slow side can sustain (stop reading from the socket until the write side drains). This is why proper streaming APIs (Node streams, Part 3) are built around backpressure, why message queues have bounded buffers (Part 10), and why "just buffer it" is a memory-exhaustion bug waiting to happen. I/O without backpressure is a leak with extra steps.

Part 2 nears its summit. We've built the OS from the privilege boundary up: syscalls, processes, threads, scheduling, concurrency, memory, files, and now I/O. Two chapters remain, and they turn from concepts to the systems you actually use: Chapter 2.8 makes Linux concrete for a working engineer (the shell, /proc, and the namespaces and cgroups that — surprise — are how containers work), and Chapter 2.9 builds virtualization and Docker on everything you now know.

Recall

• The core I/O problem: the CPU is vastly faster than devices, so the question is what it does during the wait. Interrupts let the CPU issue a request and do other work; the device signals completion by forcing the CPU into an interrupt handler. DMA moves data device↔RAM without the CPU, which is only involved at setup and the completion interrupt.
• Blocking I/O sleeps the calling thread until data is ready (simple; one thread per connection); non-blocking I/O returns immediately (complex; one thread can juggle many connections). Thread-per-connection hits the C10k problem (too many threads = memory + context-switch cost).
• I/O multiplexing lets one thread watch many sockets: select/poll rescan all descriptors each call (O(n)); epoll/kqueue register interest once and return only ready sockets (O(1) in the idle count) — the engine of the event loop (Nginx, Redis, Node.js via libuv).
• io_uring is completion-based: shared submission/completion ring buffers let an app batch thousands of I/O ops with almost no syscalls. IOPS (operations/second) is the key metric for many-small-operation (random) workloads, distinct from throughput (MB/s).
• Async/event-loop wins for I/O-bound work (mostly waiting) and is a liability for CPU-bound work (one computation blocks the loop). Zero-copy (sendfile) avoids user-space copies; backpressure prevents a fast producer from overwhelming a slow consumer.

Self-test: Why are interrupts vastly better than polling? What does DMA free the CPU from? Contrast blocking and non-blocking read(). Why does select scale badly and epoll scale well? Why does the event-loop model excel at I/O-bound work but fail at CPU-bound work?

Quiz Bank

FoundationalWhat is the difference between polling and interrupt-driven I/O?

With polling, the CPU repeatedly checks a device ("done yet?") in a loop, wasting cycles doing nothing useful during the device's slow operation. With interrupt-driven I/O, the CPU issues the request and moves on to other work; when the device finishes, it raises an interrupt that forces the CPU to jump to an interrupt handler to process the completion (e.g. wake the waiting process), then resume. Interrupts are far better because they let the CPU stay productive during the long dead-time of I/O — the hardware basis of the "Blocked" process state.

FoundationalWhat is DMA and why does it matter?

DMA (Direct Memory Access) is dedicated hardware that transfers data between a device and RAM without the CPU doing the copying. The CPU only sets up the transfer and is notified (by one interrupt) at completion; the DMA controller moves the bytes in between. It matters because without it, the CPU would be tied up copying every byte of every transfer (megabytes) instead of running programs. DMA + interrupts together are why a machine can do heavy I/O (downloads, disk reads) while the CPU stays free for real work.

AppliedWhat is the difference between blocking and non-blocking I/O, and what architecture does each imply?

Blocking I/O: a call like read() doesn't return until data is available — the thread sleeps (Blocked) meanwhile. It implies a thread-per-connection architecture (each concurrent I/O needs its own thread), which is simple but scales poorly. Non-blocking I/O: read() returns immediately, with data or a "would block" status — the thread is never put to sleep and can service other work. It implies a single-threaded event-loop architecture where one thread multiplexes many connections. The trade-off is simplicity (blocking) vs scalability to huge connection counts (non-blocking).

AppliedWhat is the C10k problem and how is it solved?

The C10k problem is handling 10,000 simultaneous connections. The naive thread-per-connection (blocking) model needs ~10,000 threads, each costing a stack (memory) and adding context-switch overhead — most just Blocked on slow network I/O, wasting resources; it doesn't scale. The solution is non-blocking I/O + I/O multiplexing (epoll/kqueue): a single thread (an event loop) watches all connections and works only on those that are ready, eliminating per-connection threads. This is how Nginx, Redis, and Node.js handle tens of thousands of connections efficiently.

InterviewWhy does select scale poorly and epoll scale well?

select/poll are stateless: on every call you pass the entire list of file descriptors, and the kernel scans all of them to find which are ready — O(n) work per call regardless of how many actually became ready, so with thousands of mostly-idle connections the linear rescan dominates (and select caps at ~1024 fds). epoll is stateful: you register interest in each fd once, and the kernel maintains a ready list as events occur; epoll_wait returns only the ready fds, costing roughly O(1) in the number of idle connections (you pay only for active ones). This makes epoll efficient for tens of thousands of connections — the core of high-performance event loops.

InterviewWhy is the single-threaded event-loop model great for I/O-bound servers but bad for CPU-bound work?

For I/O-bound work (thousands of connections mostly waiting on network/disk), one event-loop thread with non-blocking I/O + epoll handles them all: while any connection waits, the thread services others, so there's no need for thousands of mostly-Blocked threads and their memory/context-switch cost — a perfect fit. For CPU-bound work, the same single thread is a liability: any long computation blocks the entire loop, so every other connection stalls until it finishes ("don't block the event loop"). That's why such runtimes (Node.js) offload CPU-heavy work to worker threads or separate processes, keeping the loop free to handle I/O. The regime (waiting vs computing) dictates whether the model wins.

StaffYour Node.js service handles thousands of connections smoothly, but adding a CPU-heavy image-processing endpoint makes ALL requests slow whenever it's hit. Explain and fix, using this chapter.

Node runs your JavaScript on a single event-loop thread (built on non-blocking I/O + epoll/kqueue via libuv). That's ideal for the I/O-bound majority — thousands of connections mostly waiting, serviced as they become ready. But the image-processing endpoint is CPU-bound: it runs a long synchronous computation on the event-loop thread, and while it runs, the loop cannot process any other ready socket — so every connection stalls until the image is done ("blocking the event loop"). The symptom (all requests slow when that one endpoint is hit) is the signature. Fixes: (1)

move the CPU work off the loop — run it in a Node worker thread (worker_threads) or a separate process/service, so the main loop stays free for I/O; (2) for scaling, run a pool of workers (or offload to a dedicated image-processing service / queue — Part 10) and hand results back asynchronously; (3) if using native libraries, ensure they release to libuv's thread pool rather than blocking JS. The staff framing: the event-loop model's strength (one thread, no blocking) is exactly its weakness for compute — never do heavy CPU work on the loop; isolate it onto threads/processes so the loop keeps doing what it's great at (multiplexing I/O).

Flashcards

FlashPolling vs interrupts

Polling: CPU busy-checks the device (wasteful). Interrupts: device signals the CPU when done, so the CPU works on other things meanwhile.

FlashWhat DMA does

Moves data between device and RAM without the CPU; CPU only sets up and gets a completion interrupt.

FlashBlocking vs non-blocking I/O

Blocking: call sleeps the thread until ready (thread-per-connection). Non-blocking: call returns immediately (one thread multiplexes many connections).

FlashC10k problem

Handling 10,000 connections; thread-per-connection doesn't scale (memory + context switches). Solved by non-blocking I/O + epoll event loop.

Flashselect vs epoll

select: rescans all fds each call, O(n). epoll: register once, kernel returns only ready fds, ~O(1) in idle count. Engine of the event loop.

Flashio_uring

Completion-based I/O via shared submission/completion ring buffers — batch thousands of ops with almost no syscalls.

FlashIOPS

I/O operations per second — the bottleneck metric for many-small/random operations (vs throughput MB/s for few-large ones). HDD ~100s, SSD ~10,000s+.

Scenario Drill

DrillYou're choosing a cloud disk for a database that does mostly small random reads/writes. One option advertises high throughput (MB/s), another advertises high IOPS. Which matters, and why — and what if it were a video-streaming server instead?

For a database with small random operations, IOPS is the metric that matters, not throughput. Each query touches small, scattered records (2.6) — many tiny independent read/write operations — so the ceiling is how many operations per second the volume can do, regardless of their small size; a disk with huge MB/s but low IOPS will bottleneck on operation count and the database will feel slow (queue depth grows, latency spikes). So provision for high IOPS (and low latency), which is exactly why cloud providers sell IOPS as a distinct, purchasable resource and why databases favor SSDs (tens of thousands of IOPS) over HDDs (~100–200, seek-limited). For a video-streaming server, the workload inverts: it reads large sequential chunks of each video file — few, big operations — so raw throughput (MB/s) is the binding constraint, and IOPS is nearly irrelevant; you'd pick the high-throughput option (and lean on zero-copy sendfile and CDNs — Part 13 — to push bytes efficiently). The transferable judgment:

characterize your workload as many-small-random (IOPS-bound) or few-large-sequential (throughput-bound) before choosing storage — the two lead to opposite decisions and different costs. (This directly echoes the WizTree lesson from 2.6: operation count and access pattern, not total data, often decide performance.)