Appearance
7.5 — Shannon Capacity: The Ultimate Speed Limit
In 1948 Claude Shannon published A Mathematical Theory of Communication, and the field of communication changed from engineering craft to engineering science overnight.
Before it, everyone assumed that pushing data faster through a noisy channel meant accepting more errors, and that error-free communication over a noisy channel was impossible in principle. Shannon proved that is wrong. There is a rate — a specific, computable number — below which you can communicate with arbitrarily few errors, and above which you cannot communicate reliably at all.
The limit derived here is built on the noise of Chapter 7.1 and the information measure of Volume I Chapter 1.8 — the same entropy, arrived at from the opposite direction. It sets the ceiling that every scheme in Chapters 7.2 to 7.4 has been trying to approach.
1. The channel capacity theorem
\boxed{C=B\log_2\left(1+\frac SN\right)}
- C is the capacity in bits per second.
- B is the bandwidth in hertz.
- S/N is the signal-to-noise power ratio, as a plain ratio and not in decibels.
What the theorem actually claims, and both halves matter:
If your rate R is less than C, there exists a coding scheme achieving an error rate as small as you like. Not small — as small as you like. You can have one error per 10^{15} bits if you want it.
If R exceeds C, no scheme whatever can achieve reliable communication. This is not a statement about current technology; it is a proof about what is possible.
And here is the part that startled everyone in 1948: the theorem is non-constructive. Shannon proved good codes exist without producing one. He did it by showing that the average error rate over all randomly chosen codes tends to zero, so at least one code must be at least as good as the average. It took forty-five years of work to find codes that actually approach the limit.
2. Reading the formula
Bandwidth is linear
C\propto B
Double the bandwidth, double the capacity — provided the noise power does not change. But noise power does grow with bandwidth, from Chapter 7.1:
N=N_0B
So more honestly:
C=B\log_2\left(1+\frac{S}{N_0B}\right)
Doubling B doubles the first factor and reduces the argument of the logarithm. The net gain is real but less than double, and section 4 shows where it stops.
SNR is logarithmic
This is the crucial asymmetry. Capacity grows with the logarithm of SNR, so improvements become expensive fast.
At high SNR:
C\approx B\log_2\left(\frac SN\right)=\frac{B}{3.01}\cdot\text{SNR}_{dB}
Every 3 dB of extra SNR buys exactly 1 bit/s/Hz. And 3 dB means doubling the transmitter power.
Worked example. A 20 MHz channel:
| SNR | Capacity |
|---|---|
| 0 dB | 20 Mbit/s |
| 10 dB | 69 Mbit/s |
| 20 dB | 133 Mbit/s |
| 30 dB | 199 Mbit/s |
| 40 dB | 266 Mbit/s |
From 20 to 30 dB — a tenfold increase in transmitter power — bought a 50% increase in capacity. From 30 to 40 dB, another tenfold power increase, bought 34%.
That diminishing return is why nobody solves a capacity problem by turning up the power. They use more bandwidth, or more antennas, or smaller cells — all of which give linear rather than logarithmic returns.
The shape of the limit
The formula rewards bandwidth and signal-to-noise ratio very differently, and the difference is the whole engineering lesson.
Bandwidth is bought at a fair price and signal power is not. Look at the two curves. Doubling the bandwidth doubles the capacity, every time, with no diminishing return. Doubling the signal-to-noise ratio adds \log_2 2 = 1 bit per second per hertz — once. Double it again and you get one more bit. To gain another bit per hertz you must double your transmitter power, and to gain ten you must multiply it by a thousand.
That single asymmetry explains most of the last thirty years of wireless engineering. Every generation of mobile standard has chased more bandwidth — moving to higher frequencies where wide channels are available, bonding channels together, and using many antennas to create parallel channels — rather than chasing more power, which the physics prices out of reach. It is also why 5G reached for millimetre waves despite their terrible range: that is where the spare hertz are.
The flattening also sets a floor on how good a link can get. There is no amount of transmitter power that makes a narrow channel fast, which is why a 3 kHz telephone line could never carry video no matter how loud you shouted down it.
3. Worked examples
A telephone line
B = 3100 Hz, and a typical SNR of 35 dB, which is 3162 as a ratio:
C=3100\log_2(1+3162)=3100\times11.63=36{,}050\ \text{bit/s}
About 36 kbit/s — and the fastest analog telephone modems reached exactly 33.6 kbit/s. They were operating within half a decibel of the theoretical limit, which is why modem speeds stopped improving in the mid-1990s despite enormous commercial pressure.
The 56k modems that followed did not beat Shannon. They exploited the fact that the telephone network is digital internally, so a connection from an internet provider's digital equipment to a home has no analog-to-digital conversion in the downstream direction and therefore no quantisation noise in that direction. The upstream stayed at 33.6 kbit/s, and that asymmetry was the giveaway.
Wi-Fi
B=20 MHz, SNR 25 dB (316):
C=20\times10^6\times\log_2(317)=20\times10^6\times8.31=166\ \text{Mbit/s}
802.11n in a 20 MHz channel with one spatial stream achieves 72 Mbit/s. So it operates at 43% of capacity — a gap that comes from cyclic prefix overhead, pilot subcarriers, guard bands, protocol headers and the practical difficulty of the highest constellations.
With 4×4 MIMO the capacity multiplies by 4 to 664 Mbit/s, and the real system achieves about 288 Mbit/s — the same 43%.
Deep space
Voyager 1, at 24 billion kilometres, transmits 23 watts through a 3.7 m dish. The received signal is around -160 dBm.
Noise floor over a 1 Hz bandwidth with a cryogenically cooled receiver at 20 K:
P_n=10\log_{10}(kTB\times1000)=-198\ \text{dBm/Hz}
\text{SNR in 1 Hz}=-160-(-198)=38\ \text{dB}
Over a 100 Hz bandwidth the noise rises 20 dB, giving 18 dB of SNR:
C=100\log_2(1+63)=100\times6=600\ \text{bit/s}
Voyager's actual rate is 160 bit/s. Below capacity, with margin — and it uses a concatenated Reed-Solomon and convolutional code that gets within about 2 dB of the limit. Forty-eight years after launch, that link still works, and the reason it can is that the design respected this equation.
4. The infinite bandwidth limit
What if bandwidth were free? Take B\to\infty in
C=B\log_2\left(1+\frac{S}{N_0B}\right)
As B grows, S/(N_0B) becomes small, and \log_2(1+x)\approx x/\ln2 for small x:
C\to B\cdot\frac{S}{N_0B\ln2}=\frac{S}{N_0\ln2}=1.44\frac{S}{N_0}
Capacity does not grow without limit. With unlimited bandwidth it approaches a ceiling set by the ratio of signal power to noise density.
Rearranged in terms of energy per bit, with S = E_bC:
\frac{E_b}{N_0}\ge\ln2=0.693 = \boxed{-1.59\ \text{dB}}
The Shannon limit. Below -1.59 dB of E_b/N_0, reliable communication is impossible at any rate, with any code, using any bandwidth.
How close does anything get?
| Scheme | E_b/N_0 for 10^{-5} BER | Gap to limit |
|---|---|---|
| Uncoded BPSK | 9.6 dB | 11.2 dB |
| Convolutional, rate 1/2 | 4.5 dB | 6.1 dB |
| Concatenated RS + convolutional | 2.5 dB | 4.1 dB |
| Turbo codes | 0.7 dB | 2.3 dB |
| LDPC, long blocks | 0.3 dB | 1.9 dB |
| Best known LDPC | 0.0045 dB above limit | 0.005 dB |
The last row is essentially at the limit. For fifty years the gap was measured in decibels; now it is measured in thousandths of one.
What that means for the field. The era of large gains from better coding and modulation is over — there is nowhere left to go. Every improvement since about 2005 has come from somewhere else: more bandwidth (millimetre wave), more antennas (massive MIMO), smaller cells (densification), or better use of what exists (scheduling, beamforming).
5. The bandwidth-power trade
Rearranging the capacity formula in terms of spectral efficiency \eta = C/B:
\frac{E_b}{N_0}=\frac{2^\eta-1}{\eta}
This single equation is the whole design space of communication, and it is worth reading carefully.
| \eta (bit/s/Hz) | Minimum E_b/N_0 |
|---|---|
| 0.1 | -1.29 dB |
| 0.5 | -0.82 dB |
| 1 | 0 dB |
| 2 | 1.76 dB |
| 4 | 5.74 dB |
| 8 | 12.0 dB |
| 16 | 36.1 dB |
Two regimes, and every system sits in one of them.
Power-limited (\eta \lt 1). Bandwidth is plentiful, power is scarce. Use spread spectrum, low-order modulation and strong coding. Deep space probes, GPS, satellite links to small terminals, and any battery-powered sensor.
Bandwidth-limited (\eta \gt 1). Spectrum is scarce and expensive, power is available. Use high-order QAM and accept the SNR requirement. Cable television, urban mobile networks, microwave backhaul, DSL.
And the trade is genuinely favourable in one direction. Going from \eta=1 to \eta = 0.5 saves only 0.8 dB of power for half the rate; going from \eta = 8 to \eta = 4 saves 6.3 dB for half the rate. Spending bandwidth to save power works well; spending power to save bandwidth works badly — exactly the asymmetry that Chapter 7.2's FM analysis found empirically.
6. Where the formula does not apply directly
Shannon's formula assumes a flat channel with additive white Gaussian noise. Real channels are not, and the corrections matter.
Fading
A wireless channel's gain varies with time and frequency. The capacity is then a random variable, and there are two useful ways to state it.
Ergodic capacity — the average over the fading, appropriate when a transmission lasts long enough to experience many fading states.
Outage capacity — the rate that can be sustained for, say, 99% of the time. This is the practical figure, because a link that fails 10% of the time is not a link.
Fading always reduces capacity for a fixed average SNR, which follows from the logarithm's concavity: the average of the logarithm is less than the logarithm of the average.
Unless you have diversity. With several antennas or several frequencies, the probability that all of them fade simultaneously is small, and diversity recovers most of the loss. That is why MIMO and frequency hopping exist.
Frequency-selective channels
When the channel's gain varies across the band, capacity is found by integrating over frequency:
C=\int_B\log_2\left(1+\frac{S(f)|H(f)|^2}{N(f)}\right)df
And the optimal power allocation is water-filling: put more power where the channel is good and less where it is poor, as though pouring water into a container whose floor is the noise-to-gain ratio. Where the floor is above the water level, transmit nothing at all.
This is exactly what DSL does. A telephone line's attenuation rises steeply with frequency, so a DSL modem measures the response of each of its 256 subcarriers at start-up and allocates bits to each according to its measured SNR — anywhere from 0 to 15 bits per subcarrier. The negotiation you hear when a DSL modem trains is this measurement happening.
Multi-user channels
Shannon's original theorem is for one transmitter and one receiver. The multi-user cases are far harder and some remain unsolved after seventy years.
The broadcast channel — one transmitter, many receivers — was only fully solved in 2006.
Then there is the interference channel: several transmitter-and-receiver pairs operating at once, each one's signal arriving at the others as noise. Its capacity has no known general solution.
That is not a gap in anybody's education. It is an open problem in information theory, unsolved since it was posed. And it has a direct practical consequence: interference management in a cellular network remains an engineering art, tuned by measurement and experience, rather than the application of a formula the way a single link's capacity is.
7. Why this chapter is the centre of the volume
Shannon's theorem changed what engineers were trying to do.
Before 1948, the goal was to reduce errors by transmitting more carefully — more power, cleaner circuits, better filters. Error-free communication over a noisy channel was thought to require infinite power or infinite time.
After 1948, the goal became to get close to a known number. The question stopped being "how good can we make this?" and became "how far are we from the limit?" — a question with an answer.
And the answer, for fifty years, was "several decibels away". The pursuit of that gap produced convolutional codes, Reed-Solomon codes, trellis-coded modulation, turbo codes and LDPC codes. The gap is now essentially closed.
Three consequences worth stating plainly:
1. Analog is finished for transmission. A digital system with good coding transmits information at a rate an analog system cannot approach, because analog has no way to exploit the coding theorem. The switch from analog to digital television, radio and telephony was not fashion; it was this theorem.
2. Errors are not inevitable. A well-designed digital link operating below capacity can have an error rate so low that the link fails from hardware faults long before it fails from noise. That is why a digital photograph copied a million times is identical to the original, and it is what Volume I, Chapter 1.8 develops as information theory proper.
3. Where the remaining gains are. Not in coding, which is finished. In spatial reuse, in bandwidth, and in using the channel more cleverly — which is why 5G is characterised by more antennas and smaller cells rather than by a better modulation scheme.
Shannon also gave the other half of the theory in the same paper: source coding, which says how much a message can be compressed. Together they define the two ends of every communication system — squeeze the message as small as it can go, then protect it just enough to survive the channel. Volume I, Chapter 1.8 covers the compression side.
Chapter 7.6 turns to the physical interface between a circuit and the space around it — the antenna, and the propagation that decides whether a signal arrives at all.
Every formula above, built from scratch
None of the results in this chapter are worth memorising, because each one can be rebuilt in under a minute from something simpler. What follows is that rebuilding, one result at a time, so the formula and the reason for it sit on the same page as the explanation that needed them.
Shannon capacity
\boxed{C=B\log_2\left(1+\frac SN\right)}
C=B\log_2\left(1+\frac{S}{N_0B}\right)
High SNR approximation:
C\approx\frac{B}{3.01}\cdot\text{SNR}_{dB} \qquad\text{— 3 dB buys exactly 1 bit/s/Hz}
Infinite bandwidth limit. Using \log_2(1+x)\approx x/\ln2 for small x:
C\to\frac{S}{N_0\ln2}=1.44\frac{S}{N_0}
With S=E_bC:
\frac{E_b}{N_0}\ge\ln2=-1.59\ \text{dB} \qquad\text{(the Shannon limit)}
Bandwidth-power trade:
\frac{E_b}{N_0}=\frac{2^\eta-1}{\eta}
Water-filling for a frequency-selective channel: allocate power so that (noise-to-gain ratio + allocated power) is constant across the band, transmitting nothing where the ratio already exceeds the level.
Coding gains: convolutional 5 dB, Reed-Solomon 6 dB, turbo 9–10 dB, LDPC 10–11 dB. Best LDPC codes sit within 0.005 dB of the limit.
What the next chapter fixes
The capacity formula needs a signal-to-noise ratio, and getting a signal to the far end at all is the antenna's job. Chapter 7.6 covers how a piece of metal launches a wave, what gain really means, and how to predict the received power before anything is built.