Appearance
5.4 — Audio DSP: Inside an MP3, Equalisers and Noise Cancelling
A CD track is 1.41 megabits per second. The same music as a 128 kbit/s MP3 is eleven times smaller and most people cannot tell them apart on ordinary equipment.
Something is throwing away 91% of the data and the ear does not notice. That is not a compression algorithm being clever about redundancy — it is an algorithm that knows what human hearing cannot detect, and deletes exactly that.
Everything applied here was built earlier: the filters of Chapter 5.2, the spectrum of Chapter 5.3, and the physics of hearing in Volume IV Chapter 2.4, which is what decides which parts of a sound a codec is allowed to throw away.
1. What hearing actually does
The design of every audio codec follows from four facts about the ear.
The audible range is roughly 20 Hz to 20 kHz, and the upper limit falls with age — by 50 it is typically around 14 kHz. There is no point encoding what nobody can hear.
Sensitivity is wildly uneven across frequency. The ear is most sensitive between 2 and 5 kHz, which is where speech consonants live. At 50 Hz you need about 40 dB more sound pressure to hear a tone at all than you do at 3 kHz. The threshold of hearing curve is the floor below which anything can be discarded, and it varies by 60 dB across the band.
Loudness is roughly logarithmic, which is why the decibel exists:
L = 20\log_{10}\frac{p}{p_0}, \qquad p_0=20\ \mu\text{Pa}
A 10 dB increase sounds about twice as loud, and corresponds to ten times the power.
Frequency resolution is limited, and this is the important one. The cochlea behaves as a bank of overlapping bandpass filters, the critical bands. Two tones within the same critical band are not heard as two separate tones; they interact. There are about 24 critical bands across the audible range, and their width is roughly 100 Hz below 500 Hz and about 20% of centre frequency above that.
2. Masking
A loud sound makes a quieter nearby sound inaudible. This is masking, and it is the single mechanism that makes perceptual coding possible.
Simultaneous masking. A loud 1 kHz tone raises the threshold of hearing around it. A tone at 1.1 kHz that is 20 dB quieter becomes completely inaudible — not quiet, absent. The masking curve extends further upward in frequency than downward, because of how the cochlea's travelling wave works.
Temporal masking, which is stranger. A loud sound masks quieter sounds for about 200 ms after it — that is post-masking, and it makes sense. It also masks sounds for about 20 ms before it, which is pre-masking and seems to violate causality.
It does not. Loud sounds are processed faster through the auditory pathway than quiet ones, so a loud sound's neural signal overtakes the quiet sound's. The masking happens in the brain, not in the air.
3. How an MP3 encoder works
MP3 is properly MPEG-1 Audio Layer III, standardised in 1993, developed largely at Fraunhofer IIS in Germany with Karlheinz Brandenburg leading. The encoding chain has six stages.
Stage 1 — split into 32 subbands. A polyphase filter bank divides the spectrum into 32 equal-width bands. This is a multirate filter bank exactly as described in Chapter 5.2, and each band is decimated by 32 so the total sample count is unchanged.
Stage 2 — refine with an MDCT. Each subband is further split by a modified discrete cosine transform into 18 finer bands, giving 576 frequency lines. The MDCT is the DCT variant of Chapter 5.1 with an added property: consecutive blocks overlap by 50% and the overlapping halves cancel exactly on reconstruction, so there is no blocking artifact at the seams.
Stage 3 — analyse the psychoacoustics. In parallel, a 1024-point FFT of the same audio computes the masking threshold for every critical band, using the curves of section 2.
Stage 4 — allocate bits. For each band, compare the signal level with the masking threshold. Quantise just coarsely enough that the resulting quantisation noise sits exactly at the masking threshold — loud enough to save bits, quiet enough to be inaudible.
This is the central idea, and it is worth stating plainly: the encoder deliberately introduces noise, and hides it under the music.
Stage 5 — Huffman code. The quantised values are entropy-coded with variable-length codes, so common values get short codes. Volume I, Chapter 1.8 covers the coding theory; here it typically saves another 20%.
Stage 6 — pack into frames. Each frame carries 1152 samples with a header and side information. A bit reservoir lets a frame borrow unused bits from previous frames, so a difficult passage can spend more than its nominal share.
Where the bits go
At 128 kbit/s stereo, each frame of 1152 samples has 3,344 bits for two channels. That is 1.45 bits per sample, against the CD's 16. The other 14.55 bits per sample were carrying information the ear cannot use.
The artifacts, and why they sound like that
- Pre-echo. A sudden attack — a castanet, a snare rim — is spread across the whole MDCT block, so quantisation noise appears before the attack. It sounds like a brief smear or lisp. The encoder's defence is window switching: detect the transient and switch to shorter blocks for that frame, trading frequency resolution for time resolution exactly as Chapter 5.3 described.
- Birdies. At low bit rates the encoder keeps only a few of the high-frequency lines, and which ones survive changes frame to frame. The result is a warbling, watery sound in cymbals.
- Stereo collapse. Joint stereo encodes mid and side channels rather than left and right, since the side channel usually has less energy. Pushed too hard, the stereo image narrows and reverberation loses its spaciousness.
What replaced it
AAC uses a pure MDCT with better window switching, more efficient coding tools, and no 32-band prefilter — roughly the same quality at 70% of the bit rate. It is what streaming services and Apple use.
Opus merges a speech codec and a music codec, switching automatically, with very low latency. It is genuinely better than both at most rates and is the codec of every voice call over the internet.
Lossless — FLAC, ALAC — use linear prediction rather than psychoacoustics: predict each sample from the previous ones and store only the prediction error, which is small and compresses well. Typically 50 to 60% of the original size, with every bit recoverable. No perceptual model, no artifacts, and no argument about whether it is transparent.
4. Equalisers
An equaliser adjusts the level of different frequency bands, and every one is built from the biquads of Chapter 5.2.
A graphic equaliser has fixed bands — 10, 15 or 31 of them, spaced by octaves or thirds of octaves — each with a slider. Simple to use, and each band is one peaking biquad with a fixed centre frequency and Q.
A parametric equaliser gives three controls per band: centre frequency, gain, and Q (which sets the width). Far more capable, and what any serious mixing work uses.
The peaking filter's transfer function, in its standard form:
H(s)=\frac{s^2+\frac{A\omega_0}{Q}s+\omega_0^2}{s^2+\frac{\omega_0}{AQ}s+\omega_0^2}
where A=10^{G/40} for a gain of G dB. Run it through the bilinear transform of Chapter 5.2 and you get the five biquad coefficients.
The three practical filter shapes and when each is right:
- Peaking — boosts or cuts a band, leaving everything else alone. The default for fixing a resonance.
- Shelving — boosts or cuts everything above or below a corner frequency. The right shape for a broad tonal adjustment, because it does not create a bump.
- High-pass — removes everything below a corner. The most useful equaliser move there is: rolling off below 80 Hz on everything except bass and kick drum removes rumble and stage vibration and costs nothing audible.
The rule that experienced engineers apply: cutting sounds more natural than boosting. A boost adds energy the microphone never captured and emphasises whatever noise is at that frequency; a cut removes something that was genuinely a problem. And a narrow cut is nearly inaudible as a tonal change while completely removing a ringing resonance.
5. Dynamics processing
A compressor reduces the level of loud passages so the quiet ones can be raised, narrowing the dynamic range.
The controls, and what each actually does:
- Threshold — the level above which it acts.
- Ratio — how much it reduces. 4:1 means 4 dB in above threshold becomes 1 dB out.
- Attack — how quickly it responds to a level increase. Short attack catches transients and dulls them; long attack lets the initial hit through and then clamps down, which is what makes a drum sound punchy.
- Release — how quickly it lets go. Too short causes audible pumping; too long makes the quiet parts stay suppressed.
- Knee — whether the ratio applies abruptly at threshold (hard knee) or eases in (soft knee).
The implementation is a level detector — usually the rms or peak, smoothed with the exponential filter of Chapter 4.3 — feeding a gain computation, feeding a variable gain stage. The attack and release times are the time constants of that smoothing filter, so the whole processor is one first-order filter and a lookup.
A limiter is a compressor with a very high ratio and a very short attack, used to prevent clipping. A look-ahead limiter delays the audio by a few milliseconds while examining it, so it can begin reducing gain before a peak arrives rather than catching up afterwards. That is a deliberately non-causal filter, legitimate because the delay is acceptable (Chapter 4.2).
The loudness war. From the mid-1990s, mastering engineers used heavier and heavier compression to make records sound louder than competitors on the radio. Dynamic range on popular releases fell from around 14 dB to under 6 dB. Streaming services ended it by normalising everything to a target loudness — a heavily compressed track now gets turned down to match, so it sounds smaller rather than louder, and the incentive reversed.
6. Reverb
Reverberation is the sum of thousands of reflections, and it is what makes a room sound like a room.
Convolution reverb takes the actual measured impulse response of a real space (measured by the maximum-length-sequence method of Chapter 5.3) and convolves the signal with it. Perfectly accurate, and expensive — a 3-second reverb at 48 kHz is a 144,000-tap FIR filter, needing the FFT-based fast convolution of Chapter 5.1 to be feasible at all.
Algorithmic reverb builds an approximation from delay lines and all-pass filters. Manfred Schroeder's 1961 design — a bank of comb filters in parallel feeding a chain of all-pass filters — is still the skeleton of most reverbs, and it costs a thousandth of what convolution does. It sounds less like a specific real room and often more pleasing, which is why both kinds are still sold.
Reverberation time T_{60} is the time for the sound to fall by 60 dB. Sabine's formula, from measurements he made in a Harvard lecture hall in the 1890s:
T_{60}=\frac{0.161V}{A}
with V the room volume in cubic metres and A the total absorption in square metres of open-window equivalent.
Typical values: a living room 0.4 s, a concert hall 1.8 to 2.2 s, a cathedral 6 to 10 s. A recording studio's live room is around 0.5 s and its control room around 0.3 s, because you must hear what the recording contains rather than what the room adds.
7. Noise cancelling
The most impressive everyday application of the material in Part 4.
The principle is destructive interference. Measure the unwanted sound, invert it, and play it into the ear. The two add to zero.
y(t)=n(t)+\left[-n(t)\right]=0
Trivial in principle, and hard in three specific ways.
Feedforward. A microphone on the outside of the earcup hears the noise before it reaches the ear, and the electronics generate the anti-noise. The problem is that the filter must model exactly how sound travels from that microphone, around the earcup, to the ear canal — including the ear's own acoustics, which differ between people.
Feedback. A microphone inside the earcup hears the residual error and drives it towards zero. This measures the actual thing you care about rather than predicting it, but it is a feedback loop and can become unstable — which is what the whole of Part 6 is about, and it is why a badly designed pair squeals.
Hybrid systems use both, and every good product does.
Why it works for engines and not for speech
The bandwidth limit is physical. To cancel a sound you must produce the anti-noise within a small fraction of a wavelength of the arrival point, and the electronics must respond within a fraction of a period.
At 100 Hz the wavelength is 3.4 m and the period is 10 ms — enormous tolerances, and cancellation of 25 to 30 dB is routine.
At 2 kHz the wavelength is 17 cm and the period 0.5 ms. The microphone-to-driver distance is now a significant fraction of a wavelength, and the processing delay is a significant fraction of a period. Cancellation drops to a few dB.
Above about 1 kHz, passive isolation — sealing the ear — works far better than any active system, which is why good noise-cancelling headphones also have well-sealed cups. The active system handles the low rumble that no practical seal can block; the passive seal handles the high frequencies that no practical electronics can cancel.
And the deep reason the aeroplane case works so well: cabin noise is dominated by low-frequency engine and airflow rumble, which is precisely the band where active cancellation is effective. Speech, at 300 Hz to 3 kHz, mostly is not cancelled — which is why you can still hear the cabin announcement.
The adaptive filter
The transfer function from the speaker to the ear changes with fit, with head movement, with whether you are wearing glasses. So the filter must adapt.
The LMS (least mean squares) algorithm updates the coefficients continuously:
w[n+1]=w[n]+\mu\,e[n]\,x[n]
Read it in words: adjust each coefficient in proportion to the error, in the direction that reduces it. \mu is the step size, and it is the whole design trade — large \mu adapts fast and risks instability, small \mu is stable and slow to respond.
The same algorithm, unchanged, does echo cancellation on every phone call, adaptive equalisation in every modem, and beamforming in every smart speaker's microphone array. It is one of the most reused pieces of mathematics in engineering, and Volume II, Chapter 10.4 develops the gradient-descent reasoning behind it.
Chapter 5.5 applies the same transforms to pictures, where the axes are spatial rather than temporal and the compression story runs almost in parallel.
Every formula above, built from scratch
None of the results in this chapter are worth memorising, because each one can be rebuilt in under a minute from something simpler. What follows is that rebuilding, one result at a time, so the formula and the reason for it sit on the same page as the explanation that needed them.
Audio
L_{dB}=20\log_{10}\frac{p}{p_0}, \qquad p_0=20\ \mu\text{Pa}
Critical bandwidth (the Bark scale approximation):
\text{BW}\approx25+75\left[1+1.4(f/1000)^2\right]^{0.69} \ \text{Hz}
About 100 Hz below 500 Hz, and roughly 20% of centre frequency above that.
MP3 frame: 1152 samples. At 128 kbit/s and 44.1 kHz:
\text{bits per frame}=\frac{128{,}000\times1152}{44{,}100}=3344
\text{bits per sample per channel}=\frac{3344}{1152\times2}=1.45
against 16 for uncompressed — a ratio of 11:1.
Sabine's reverberation formula:
T_{60}=\frac{0.161\,V}{A}
V in m³, A in m² of absorption.
LMS adaptive filter update:
w[n+1]=w[n]+\mu\,e[n]\,x[n]
Stability condition:
0 \lt \mu \lt \frac{2}{\lambda_{max}} \approx \frac{2}{(M+1)P_x}
for M+1 taps and input power P_x. Larger \mu adapts faster and converges less accurately; smaller \mu is the reverse.
What the next chapter fixes
Sound is one number changing over time. An image is millions of numbers arranged in a plane, and every operation in this Part has a two-dimensional twin. Chapter 5.5 works through them, and ends at the operation that turned out to be the foundation of modern machine vision.