5.4.1 — Ports, Multiplexing & UDP

Chapter 5.3 delivered a packet to a machine. That machine is running a browser, a music player, two terminal sessions, a database, a backup agent and eleven background services.

Which one gets the packet?

The IP address answered "which computer". Nothing so far answers "which program". That is the transport layer's first job, and the answer is a 16-bit number called a port.

1. Ports: the flat number and the apartment building

An IP address is the street address of a building. A port is the apartment number.

The postman delivers to the building; the number on the envelope decides which door it goes through. Two apartments in the same building can both be expecting post, and nothing gets confused, because the two numbers together — building plus apartment — identify exactly one destination.

A port is 16 bits, so 0–65,535. It lives in the transport header, and there are two of them in every packet:

┌─────────────────┬─────────────────┬──────────────
│ source port     │ dest port       │  the rest
│    16 bits      │    16 bits      │
└─────────────────┴─────────────────┴──────────────

The destination port says which program should receive this. The source port says which program sent it, so the reply knows where to go back to.

The ranges are conventional and worth knowing:

Range	Name	Meaning
0–1023	well-known	assigned by IANA; on Unix, binding one requires root
1024–49151	registered	claimed by specific applications, loosely enforced
49152–65535	ephemeral	handed out temporarily to clients

The ports you will actually meet:

Port	Service	Port	Service
22	SSH	443	HTTPS
25	SMTP (mail)	3306	MySQL
53	DNS	5432	PostgreSQL
80	HTTP	6379	Redis
123	NTP (time)	27017	MongoDB

Why root is required below 1024 is a genuine security decision from early Unix, not an arbitrary rule. If any user could bind port 22, an ordinary user could run a fake SSH server on the machine and harvest everyone's passwords. Restricting the well-known ports to root means that if you connect to port 22 on a machine, you know the administrator put it there. It is a weak guarantee by modern standards, and it is why containers and modern services usually run on a high port behind a proxy that owns port 443 (Chapter 9.9.7).

2. The four-tuple: what actually identifies a connection

Here is the question people get wrong. Your browser has six tabs open to the same server. All six connect to the same destination IP and the same port 443. How does the operating system tell the six replies apart?

Not by destination port — that is 443 for all of them. The answer is that a connection is identified by four values together:

(\text{source IP},\ \text{source port},\ \text{destination IP},\ \text{destination port})

This is the four-tuple, and it is the key in the kernel's connection table.

Each tab is given a different ephemeral source port — 51000, 51001, 51002 and so on — chosen by the operating system when the connection opens. So the four tuples differ in exactly one component, and that is enough:

(192.168.1.42, 51000, 142.250.187.238, 443)   → tab 1
(192.168.1.42, 51001, 142.250.187.238, 443)   → tab 2
(192.168.1.42, 51002, 142.250.187.238, 443)   → tab 3

Two consequences fall out of this immediately, and both matter in production.

A server on one port handles enormous numbers of simultaneous connections. A web server listening on port 443 does not need a port per client. Every client brings its own source IP and source port, so the tuples are naturally distinct. The famous "C10k problem" — ten thousand concurrent connections on one machine (Chapter 2.7) — was never about running out of ports on the server. It was about the cost of the operating system managing that many sockets.

A client can run out of ports, and this is a real outage cause. One machine making many outbound connections to the same destination is limited by its ephemeral port range, because the other three tuple components are fixed. With a default Linux range of roughly 28,000 ports, a service opening a fresh connection per request to one backend hits the ceiling at around 28,000 concurrent connections — and worse, sockets sit in a TIME_WAIT state for a minute or two after closing (Chapter 5.4.2), so the ports are not immediately reusable.

The symptom is EADDRNOTAVAIL or "cannot assign requested address" under load. The fix is almost never to widen the port range. It is connection pooling — reuse a small number of long-lived connections instead of opening one per request. That is exactly why HTTP keep-alive exists (Chapter 5.6) and why every database driver ships a connection pool (Chapter 7.2).

3. The transport layer's second job, and the choice it forces

Multiplexing by port is the job both transport protocols do. Everything else is a choice, and there are exactly two answers shipped at scale.

UDP does nothing else. It adds ports, a length and a checksum, and hands your data to IP. If the packet is lost, it is lost. If two arrive out of order, they arrive out of order. Nobody is told.

TCP does everything else. It turns IP's unreliable, unordered, no-promises delivery into an ordered stream of bytes that does not lose data, does not duplicate it, and slows down when the network or the receiver cannot keep up.

Chapter 5.1's framing applies exactly here: IP said "I might lose your packet", and TCP is the layer that answers that limitation. UDP is the layer that declines to.

4. UDP, in full — it really is this small

The entire UDP header is eight bytes:

┌──────────────┬──────────────┬──────────────┬──────────────┐
│ source port  │  dest port   │    length    │   checksum   │
│   16 bits    │   16 bits    │   16 bits    │   16 bits    │
└──────────────┴──────────────┴──────────────┴──────────────┘

Source and destination port — the multiplexing from section 1.

Length — the size of the header plus data. Slightly redundant, since IP also carries a length, and it is a fossil.

Checksum — covers the header and the data. If it fails, the datagram is dropped. Note it was optional in IPv4 and is mandatory in IPv6.

That is the whole protocol. There is no connection, no acknowledgement, no sequence number, no retransmission, no ordering, no flow control and no congestion control. The unit is called a datagram rather than a segment or a stream, and the word is precise: it is a self-contained message, like a telegram, with no relationship to the one before or after it.

What UDP gives you that TCP cannot:

Message boundaries are preserved. Send three datagrams of 100 bytes and the receiver gets exactly three reads of 100 bytes. TCP does not do this — it is a byte stream, so three 100-byte writes may arrive as one 300-byte read, or as a 250-byte read followed by a 50-byte read. Section 5 covers why that matters more than people expect.

No head-of-line blocking. If datagram 5 is lost, datagram 6 is still delivered immediately. TCP would hold 6 back until 5 arrived, because it promised ordering.

No connection setup. The first packet carries data. TCP needs a round trip before any data moves (Chapter 5.4.2).

One-to-many is possible. UDP can broadcast and multicast; TCP is strictly point-to-point, because a connection has exactly two ends.

5. When UDP is the right answer

The rule people repeat is "UDP is for when speed matters more than reliability", and that is a poor summary because it suggests you are trading correctness for speed. The better rule:

Use UDP when late data is worse than no data, or when you can build a better reliability scheme than TCP's for your specific case.

Six real cases, each with the reason.

DNS (Chapter 5.5). A query and a response are one small packet each. TCP's three-packet handshake to exchange two packets would triple the cost. And the recovery from loss is trivial: ask again. When a retry is cheaper than a connection, do not build a connection.

Live audio and video. A voice packet from 400 ms ago is useless — the conversation has moved on. Retransmitting it wastes bandwidth and delays everything behind it. Dropping it and playing a tiny gap is genuinely the correct behaviour, and it is why a bad video call goes blocky rather than pausing.

Online games. The same argument, sharper. A player-position update from three frames ago is worse than useless — applying it would teleport the character backwards. The game sends the current position and lets old ones die.

NTP (time synchronisation). The whole point is to measure how long a round trip takes. TCP's retransmission and buffering would corrupt the measurement.

QUIC and HTTP/3 (Chapter 5.4.3). This is the interesting one and the reason "UDP is unreliable" is a misleading summary. QUIC runs over UDP and is fully reliable — it implements acknowledgements, retransmission, ordering and congestion control itself. It uses UDP not to avoid reliability but to escape TCP, because TCP is implemented in operating system kernels and in middleboxes across the internet, and therefore cannot be changed (the ossification from Chapter 5.1). UDP here is not "the unreliable option". It is "the option that lets me write my own transport in user space."

High-volume metrics and logs. A monitoring agent sending a hundred thousand counters per second does not care if 0.1% are lost, and absolutely does care about not blocking on a slow collector.

Where UDP is the wrong answer: anything where every byte must arrive and order matters. File transfer, a database protocol, an API request, email. If you find yourself adding acknowledgements and retransmission to a UDP application, stop — you are reimplementing TCP, and you will do it worse, because TCP's congestion control alone represents forty years of accumulated fixes.

6. The trap nobody warns you about: TCP has no message boundaries

This is the single most common bug when someone writes network code for the first time, and it belongs here because UDP is where the contrast is clearest.

// A client sends three separate writes
socket.write('{"cmd":"ping"}');       // (1)
socket.write('{"cmd":"status"}');
socket.write('{"cmd":"quit"}');

1. Three calls to write. It is extremely tempting to assume the server will get three data events with one JSON object each.

It will not, reliably. TCP is a byte stream. The three writes may be coalesced into one segment by the sender's kernel, split across segments by the maximum segment size, or delivered in any grouping the network happens to produce. The server might see:

{"cmd":"ping"}{"cmd":"status"}{"cmd":"q          ← one event, and a partial object
uit"}                                             ← the rest, later

The bug is invisible in testing, because on localhost with small messages the writes usually do arrive one per read. It appears in production, under load, with larger payloads, and looks like random JSON parse errors.

The fix is framing: you must define your own message boundaries, and there are exactly three ways.

Length prefixing. Write the length first, then the body. The reader reads the length, then reads exactly that many bytes. This is what HTTP's Content-Length does, and what 4.14.1's Encode and Decode Strings problem was teaching. It is the most robust option because the body may contain any byte.

A delimiter. Terminate each message with a byte that cannot appear inside one — a newline for line-based protocols. Simple, but it forces you to escape the delimiter in the data, and a message that is missing its terminator hangs the reader forever.

Self-describing structure. The message format itself says where it ends, as with a JSON parser that counts braces. Works, but requires parsing as you read.

// Length-prefixed framing over a TCP socket
let buffer = Buffer.alloc(0);                              // (1)

socket.on('data', (chunk: Buffer) => {
  buffer = Buffer.concat([buffer, chunk]);                 // (2)
  while (buffer.length >= 4) {                             // (3)
    const length = buffer.readUInt32BE(0);                 // (4)
    if (buffer.length < 4 + length) break;                 // (5)
    const message = buffer.subarray(4, 4 + length);        // (6)
    handle(JSON.parse(message.toString()));
    buffer = buffer.subarray(4 + length);                  // (7)
  }
});

1. A persistent buffer holding whatever has arrived but not yet been consumed. This variable is the whole solution — you cannot process a data event in isolation.
2. Append the new chunk to whatever was left over.
3. A while, not an if. One chunk may contain several complete messages, and stopping after one would leave the rest unprocessed until the next chunk arrives — which may never come.
4. Read the 4-byte length header. BE is big-endian, the network byte order convention (Chapter 5.9 explains why).
5. The full body has not arrived yet. Leave everything in the buffer and wait for more data.
6. Extract exactly one message.
7. Discard the consumed bytes. Forgetting this line is a memory leak that grows for the life of the connection.

With UDP none of this is needed, because a datagram is a message by definition. One send produces one recv, always. That property alone is sometimes the reason to choose UDP.

What the interviewer will push on

"What is a port, and how does a server handle 10,000 connections on one port?" A port identifies a program on a machine. The server does not need more ports, because a connection is keyed on the four-tuple and each client brings a distinct source IP and source port. The tell is naming the four-tuple rather than saying "the operating system handles it".

"Your service throws 'cannot assign requested address' under load. Diagnose it." Ephemeral port exhaustion on the client side: many outbound connections to one destination, so three of the four tuple components are fixed. Made worse by TIME_WAIT holding ports after close. The fix is connection pooling, not a wider port range — and saying that is what separates a diagnosis from a workaround.

"When would you choose UDP over TCP?" Not "when speed matters". When late data is worse than no data (live media, game state, time sync), when a retry is cheaper than a connection (DNS), or when you need to implement your own transport (QUIC). Then note that QUIC is fully reliable over UDP, which shows you understand UDP is a blank slate rather than an unreliable option.

"You send three messages over TCP and the receiver gets two events. Why?" TCP is a byte stream with no message boundaries. You must frame — length prefix, delimiter, or self-describing format. This is asked because it is the bug everyone writes once, and because the answer reveals whether you have actually written socket code.

"Why does binding to port 80 need root on Linux?" So an unprivileged user cannot impersonate a standard service on that machine. Then note that modern deployments avoid it entirely by running on a high port behind a proxy or a load balancer that owns 443.

One thing to volunteer: point out that UDP preserving message boundaries is a feature, not just an absence of TCP's guarantees — and that it is occasionally the deciding reason to pick it, independent of any reliability argument. Most candidates only frame UDP as "TCP minus things".

Recall

• A port is 16 bits and identifies a program; the IP address identified the machine. Below 1024 needs root on Unix, so a standard service cannot be impersonated by an ordinary user.
• A connection is keyed on the four-tuple (source IP, source port, destination IP, destination port). That is why one server port serves 10,000 clients — and why a client making many connections to one destination hits ephemeral port exhaustion, whose fix is connection pooling, not a wider range.
• UDP is eight bytes: two ports, a length and a checksum. No connection, no acknowledgements, no ordering, no retransmission, no congestion control.
• What UDP gives you: preserved message boundaries, no head-of-line blocking, no setup round trip, and one-to-many delivery.
• Choose UDP when late data is worse than no data, when a retry is cheaper than a connection, or when you are writing your own transport — QUIC is fully reliable and runs over UDP purely to escape TCP's ossification.
• TCP is a byte stream with no message boundaries. Three writes may arrive as one read or as a partial one, so you must add framing: length prefix, delimiter, or self-describing structure. The persistent leftover buffer and the while loop are both mandatory.

Self-test: How does one server port distinguish six tabs from the same browser? · What error appears on ephemeral port exhaustion, and what is the real fix? · Give three reasons to pick UDP that are not "it is faster" · Why is QUIC over UDP not an argument that UDP is unreliable? · Why must the framing loop be a while and not an if?

Next: 5.4.2 builds the other answer — how TCP turns a network that loses, duplicates and reorders packets into a stream that behaves as if none of that ever happens, starting with the three packets that open every connection.