Appearance
4.7 — The Sampling Theorem and Aliasing
A CD stores 44,100 numbers per second per channel. Between one number and the next there is nothing — no record of what the sound was doing for those 22.7 microseconds. And yet the music comes back complete, with nothing missing.
That should be impossible. The sampling theorem explains why it is not, and it puts an exact condition on when it works.
Everything here depends on Chapter 4.4's result that a signal is a sum of frequencies, because the whole failure being described is what happens when two different frequencies become indistinguishable once you only look at a signal at intervals.
1. The theorem
A signal containing no frequency higher than B hertz is completely determined by its samples taken at any rate greater than 2B samples per second.
The rate 2B is the Nyquist rate. Half the sampling rate, f_s/2, is the Nyquist frequency — the highest frequency that a given sampling rate can represent.
"Completely determined" is the strong claim, and it means exactly what it says: the original continuous signal can be reconstructed perfectly, not approximately, from the samples alone. Nothing is lost.
Harry Nyquist established the rate at Bell Labs in 1928 while studying telegraph transmission. Claude Shannon proved the reconstruction result in 1949. Vladimir Kotelnikov had it in Russia in 1933 and E. T. Whittaker had the interpolation formula in 1915, which is why the theorem carries four names depending on which textbook you read.
2. Why it is true
The proof is short once the Fourier transform is available, and it makes the aliasing mechanism obvious rather than mysterious.
Step 1 — model sampling as multiplication. Sampling is multiplying the signal by an impulse train, a row of impulses spaced T_s apart:
p(t)=\sum_{n=-\infty}^{\infty}\delta(t-nT_s)
x_s(t)=x(t)p(t)=\sum_n x(nT_s)\delta(t-nT_s)
Each impulse sifts out one sample value.
Step 2 — the impulse train's own transform is another impulse train. This is the key fact, and it is worth stating plainly: a row of impulses spaced T_s apart in time transforms into a row of impulses spaced f_s = 1/T_s apart in frequency.
P(f)=f_s\sum_{k=-\infty}^{\infty}\delta(f-kf_s)
The spacings are reciprocal, which is the time-bandwidth trade of Chapter 4.4 in its most extreme form: closer together in time means further apart in frequency.
Step 3 — multiplication in time is convolution in frequency. By the convolution theorem:
X_s(f)=X(f)*P(f)=f_s\sum_{k}X(f-kf_s)
Read that result. Convolving with an impulse at kf_s shifts the spectrum to be centred there. So:
\boxed{\text{Sampling makes infinitely many copies of the spectrum, spaced } f_s \text{ apart.}}
Step 4 — the condition. The original spectrum occupies -B to +B, a total width of 2B. The copies are spaced f_s apart. They avoid overlapping if and only if f_s > 2B.
If they do not overlap, a low-pass filter passing only -f_s/2 to +f_s/2 removes every copy but the original, and the signal is recovered exactly. That is the theorem.
If they do overlap, the overlapping regions have added together. Addition is not invertible — given the sum you cannot know the two parts. The damage is done at the instant of sampling and no processing afterwards can undo it.
3. Aliasing, concretely
An overlapping high frequency does not vanish; it reappears somewhere else in the band, disguised as a lower frequency:
f_{alias}=\left|f-k f_s\right| \quad\text{for whichever integer } k \text{ lands it in } 0 \text{ to } f_s/2
Worked example. Sample at 8 kHz. Nyquist is 4 kHz.
| Input | Alias | What you hear |
|---|---|---|
| 3 kHz | 3 kHz | correct |
| 5 kHz | 3 kHz | wrong, and indistinguishable |
| 11 kHz | 3 kHz | wrong |
| 13 kHz | 3 kHz | wrong |
Four different tones all become 3 kHz. There is no clue in the samples that anything went wrong — the data looks perfectly reasonable.
The everyday sightings
Wagon wheels turning backwards. Film runs at 24 frames per second. A wheel with 12 spokes turning at 2 revolutions per second presents a spoke at the same position 24 times a second — exactly the frame rate — so it looks stationary. Slightly slower and it appears to creep backwards.
Aircraft propellers on a phone camera. The rolling shutter samples different rows of the sensor at slightly different times, producing bent, broken blades that never existed.
Moiré patterns. A finely striped shirt on television, or a photograph of a computer screen. The camera's pixel grid is a spatial sampler, and the stripe frequency exceeds its Nyquist limit, producing large false patterns. This is aliasing in two spatial dimensions rather than in time, and it is the reason cameras historically had an optical low-pass filter glued in front of the sensor — a deliberate slight blur to prevent it.
A stroboscope, which is the same effect used deliberately. Flash a light at nearly the rotation rate of a machine and it appears frozen or slowly turning, letting you inspect it while it runs.
4. The anti-alias filter
Because the damage happens at sampling, the filter must come before the sampler. This is not negotiable and it is the most common mistake in a first data acquisition design.
The filter must pass the band of interest and remove everything above f_s/2. As Chapter 3.5 computed, doing that with a narrow transition band requires an impractically steep analog filter.
The universal solution is oversampling. Sample at 8, 64 or 256 times the required rate, so the analog filter has a huge transition band to work with and can be a gentle two-pole design. Then filter digitally, where a steep response is just arithmetic, and discard the surplus samples.
The other reason to oversample: every doubling of the rate gains 3 dB of signal-to-noise ratio in the band of interest, because the fixed quantisation noise power is spread over a wider band and you keep only part of it. Chapter 3.5's sigma-delta converters take this to its logical conclusion.
5. Reconstruction
Going the other way: given the samples, produce the continuous signal.
The theorem says a perfect low-pass filter does it. And a perfect low-pass filter's impulse response is a sinc (Chapter 4.4), so reconstruction is:
x(t)=\sum_{n=-\infty}^{\infty}x[n]\,\text{sinc}\!\left(\frac{t-nT_s}{T_s}\right)
This is the Whittaker-Shannon interpolation formula, and it deserves a moment's attention because it explains the whole thing.
Each sample is replaced by a sinc pulse centred on it and scaled by its value. A sinc is 1 at its centre and exactly zero at every other sample instant. So at each sampling moment, exactly one sinc contributes its full value and every other contributes precisely nothing — the reconstruction passes through every sample exactly. And between the samples, the overlapping tails of all the sincs add up to exactly the right curve.
Why perfect reconstruction is impossible in practice. The sinc extends infinitely in both directions, so computing the value at any instant needs every sample, including future ones. That makes the ideal reconstructor both infinite and non-causal.
What real converters do instead is hold each sample constant until the next — a zero-order hold, which is a staircase. Its frequency response is a sinc:
H(f)=T_s\,\text{sinc}(fT_s)
which droops towards Nyquist, reaching -3.9 dB at f_s/2. That droop is real and audible, so converters apply a compensating boost in the digital filter, and the specification sheets call it "sinc compensation" or "\sin x/x correction". Oversampled converters largely avoid the problem, because the droop over the small fraction of the band actually used is negligible.
6. Undersampling, used deliberately
Aliasing is a fault when it is accidental. Done on purpose it is a technique.
A radio signal occupying 100.0 to 100.2 MHz has a bandwidth of 200 kHz, even though its frequencies are around 100 MHz. Nothing in the theorem's proof required the band to start at zero — only that the copies do not overlap.
So sample at 500 kHz. The copies land every 500 kHz, and the 100 MHz band aliases down to somewhere in the baseband, intact. You have shifted a 100 MHz signal down to audio-rate sampling without any mixer.
The condition is that the band must fit entirely within one gap between copies, which requires
\frac{2f_H}{k} \le f_s \le \frac{2f_L}{k-1}
for some integer k, where f_L and f_H are the band edges.
This is bandpass sampling or undersampling, and it is the foundation of software-defined radio. The catch is that the sample-and-hold circuit must have an analog bandwidth reaching to 100 MHz even though it samples at 500 kHz, and its aperture jitter requirement is set by the 100 MHz signal, not the sample rate — which is Chapter 3.5's jitter calculation with a very demanding input.
7. The rates the world actually uses, and why
| Rate | Where | Reason |
|---|---|---|
| 8 kHz | telephone | 3.4 kHz speech band, 4 kHz Nyquist with margin |
| 44.1 kHz | CD | 20 kHz hearing limit, plus filter transition band |
| 48 kHz | video and professional audio | divides evenly into video frame rates |
| 96, 192 kHz | high-resolution audio | easier filters, and marketing |
| 1–10 MHz | oscilloscopes and instruments | signal bandwidth |
| 2 GHz+ | radar and optical links | signal bandwidth |
The 44.1 kHz number has a genuinely odd origin. Early digital audio was stored on video tape, because that was the only medium with enough bandwidth. A video frame held a fixed number of lines with three samples per line, and 245 \times 60 \times 3 = 44{,}100 for the NTSC format — and, by coincidence of the two standards, 294\times50\times3 also equals 44,100 for PAL. The number was chosen to be compatible with both television systems, not for any acoustic reason, and it has outlived the video recorders by four decades.
Why 48 kHz won in professional work: it divides evenly by video frame rates, so a whole number of samples fits in each frame and audio stays synchronised with picture without resampling.
Whether 96 kHz is audible is a genuine argument. The honest engineering case is that it moves the anti-alias filter's transition band well above hearing, so the filter can be gentle and its phase distortion is out of the audible range. The claim that reproducing 40 kHz tones matters directly is much weaker, since human hearing stops around 20 kHz and falls with age.
8. Compressed sensing, and the limits of the theorem
The sampling theorem gives a sufficient rate, and for a long time this was read as a hard floor. Since about 2004 it has been known that for signals with additional structure, far fewer samples can suffice.
The idea turns on one property, called sparsity. A signal is sparse if, written in some suitable basis, almost all of its components are zero and only a few are not. A photograph is sparse in this sense: expressed as wavelets rather than as pixels, most of the coefficients are near zero. Crucially, you do not need to know which few are non-zero.
When a signal is sparse, the number of measurements you need is set by how many non-zero components there are, not by the signal's bandwidth. Take that many random measurements, then solve for the sparsest signal consistent with them, and you recover the original exactly. That is a far smaller number than Nyquist demands, and it is why the idea caused such a stir when Candès, Romberg, Tao and Donoho published it in 2006.
This is not a violation of the sampling theorem. The theorem assumes nothing about the signal beyond its bandwidth; compressed sensing assumes sparsity, which is a strong extra assumption. When it holds, the extra knowledge substitutes for measurements.
Where it is used: MRI scanners, where fewer measurements means a shorter scan and a patient who has to stay still for less time; some astronomical imaging; and single-pixel cameras. The mathematics is developed in Volume II, Chapter 10.4's optimisation framework.
Part 4 has built the language: signals, LTI systems, convolution, and the three transforms. Part 5 uses it, taking the discrete Fourier transform from a definition to an algorithm fast enough to have changed the world.
Every formula above, built from scratch
None of the results in this chapter are worth memorising, because each one can be rebuilt in under a minute from something simpler. What follows is that rebuilding, one result at a time, so the formula and the reason for it sit on the same page as the explanation that needed them.
Sampling
f_s \gt 2B \qquad\text{(Nyquist rate)}, \qquad f_N = f_s/2 \qquad\text{(Nyquist frequency)}
X_s(f)=f_s\sum_kX(f-kf_s)
Derivation: sampling multiplies by an impulse train; multiplication in time is convolution in frequency; convolving with impulses spaced f_s apart makes copies spaced f_s apart.
f_{alias}=|f-kf_s|
Reconstruction:
x(t)=\sum_nx[n]\,\text{sinc}\!\left(\frac{t-nT_s}{T_s}\right)
Why it works exactly: each sinc is 1 at its own sample instant and exactly zero at every other one, so the sum passes through every sample and the overlapping tails supply the correct values between them.
Zero-order hold response:
H(f)=T_s\,\text{sinc}(fT_s), \qquad \text{droop at } f_s/2 = -3.9\ \text{dB}
Bandpass sampling condition:
\frac{2f_H}{k}\le f_s\le\frac{2f_L}{k-1} \quad\text{for some integer } k
Oversampling gain:
\Delta\text{SNR}=10\log_{10}(\text{OSR}) \ \text{dB} = 3\ \text{dB per doubling}
Fourteen worked problems next.
What the next chapter fixes
Part 4 has built the mathematics. Part 5 spends it: the algorithm that made frequency analysis fast enough to run in real time, the two families of digital filter and how to choose between them, and what all of it looks like when the signal is music, speech or a photograph.