Appearance
3.5 — Entropy, Twice Over
Shuffle a new deck of cards and you will never shuffle it back into order. Nothing forbids it. Every particular arrangement of 52 cards is exactly as likely as every other, including the factory order, and the shuffle does not know or care which one it produces. There is no force pushing towards disorder.
And yet it never happens, and you would be right to bet your house on it. The reason is arithmetic, not physics: there are 52! \approx 8\times10^{67} orderings and only one of them is the factory order. Ordered arrangements are not forbidden; they are outnumbered.
Entropy is the quantity that turns that observation into a law. This chapter builds it twice — once from heat and temperature, the way Clausius did it in the 1850s with no molecular picture at all, and once from counting arrangements, the way Boltzmann did it in the 1870s. The two definitions look nothing alike and they give the same number, which is one of the deepest agreements in physics.
Definition one: heat divided by temperature
Chapter 3.4 found that for a reversible engine, Q_C/Q_H = T_C/T_H. Rearrange it:
\frac{Q_H}{T_H} = \frac{Q_C}{T_C}
Something is conserved around a Carnot cycle, and it is not heat. It is heat divided by temperature. Clausius took that hint seriously, adopted the sign convention that heat in is positive and heat out is negative, and wrote:
\frac{Q_H}{T_H} + \frac{-Q_C}{T_C} = 0
The sum of Q/T around a reversible cycle is zero. Any cycle at all can be approximated by a mesh of tiny Carnot cycles, so the result generalises:
\oint \frac{dQ_{\text{rev}}}{T} = 0
which reads: the closed-loop integral of dee-Q-reversible over T is zero. The circle on the integral sign means "all the way around a closed path".
That is exactly the property that defines a state function. If going around any loop gives zero, then the value going from A to B is the same no matter which path you take, and you can attach a number to each state. Height above sea level works this way: walk any route from your door to the summit and the altitude gained is the same. Distance walked is not a state function; altitude is.
So define entropy S by:
\boxed{\Delta S = \int_A^B \frac{dQ_{\text{rev}}}{T}}
with units of joules per kelvin. The subscript "rev" is not decoration. To compute the entropy change between two states, you must use a reversible path connecting them. But since S is a state function, the answer applies to any path between those same two states, reversible or not. That is the trick that makes every calculation below possible: invent a convenient reversible path, compute along it, and the answer is valid for the messy real process.
Why the second law becomes an inequality
For a reversible process, \Delta S_{\text{universe}} = 0: whatever entropy one body loses, another gains exactly.
For a real, irreversible process, more entropy is created. Clausius proved this from the engine argument of Chapter 3.4, and the statement is:
\boxed{\Delta S_{\text{universe}} \geq 0}
with equality only for a reversible process. This is the second law in its most useful form. No longer a prohibition on machines — a number that never decreases.
Two clarifications that head off the usual confusions.
The entropy of a part of the universe can absolutely fall. Water freezing into ice becomes more ordered and its entropy drops. A seed grows into a tree. You clean your room. None of these violate anything, because each one dumps more entropy into the surroundings than it removes from itself. The ledger that must balance is the total.
Entropy is not "disorder", quite. Disorder is a helpful first image and it misleads in specific cases. The sharper statement is that entropy counts how many microscopic arrangements are consistent with what you can see, and the counting version below makes that precise.
Computing entropy for real processes
Heating something up
Warm a substance from T_1 to T_2. Add heat slowly so the process is reversible, with dQ = mc\,dT:
\Delta S = \int_{T_1}^{T_2}\frac{mc\,dT}{T} = mc\int_{T_1}^{T_2}\frac{dT}{T} = mc\ln\frac{T_2}{T_1}
Read it: entropy rises with the logarithm of the temperature ratio. Heating water from 300 K to 600 K adds the same entropy as heating it from 600 K to 1200 K, because both are a doubling. The log appears because the same joule of heat matters more when the temperature is low — dividing by a small T gives a bigger result.
Worked number. Heat 1 kg of water from 20 °C to 100 °C, with c = 4186:
\Delta S = (1)(4186)\ln\frac{373}{293} = 4186\times\ln(1.273) = 4186\times0.2414 = 1010\ \text{J/K}
Melting or boiling
At a phase change the temperature is constant, so T comes out of the integral and what is left is just the latent heat from Chapter 3.1:
\Delta S = \frac{Q}{T} = \frac{mL}{T}
Worked number. Melt that kilogram of ice at 273 K, with L_f = 334{,}000 J/kg:
\Delta S = \frac{334000}{273} = 1223\ \text{J/K}
And boiling it at 373 K:
\Delta S = \frac{2260000}{373} = 6059\ \text{J/K}
Look at the three numbers together: melting 1223, heating all the way from freezing to boiling 1010, boiling 6059. Boiling is by far the biggest entropy jump, and it should be — a gas has vastly more ways to arrange itself than a liquid, since its molecules are free to be anywhere in a container thousands of times larger.
A gas expanding into vacuum
Here is the case that shows why the reversible-path trick matters. Take a container split by a partition, gas on one side, vacuum on the other. Puncture the partition. The gas rushes in and fills the whole volume.
This is free expansion, and it is violently irreversible. It also does no work (there is nothing to push against) and exchanges no heat (assume insulation), so Q = 0 and W = 0 and therefore \Delta U = 0. For an ideal gas that means the temperature is unchanged.
Now: is \Delta S = 0, since Q = 0?
No — and this is the trap. The formula requires dQ_{\text{rev}}, and this process was not reversible. So invent a reversible path between the same two states. The gas goes from volume V_1 to V_2 at the same temperature, so use a slow isothermal expansion, which from Chapter 3.3 has Q = W = nRT\ln(V_2/V_1):
\Delta S = \frac{Q_{\text{rev}}}{T} = nR\ln\frac{V_2}{V_1}
For a doubling, \Delta S = nR\ln 2 = 5.76 J/K per mole. Positive, and since nothing else in the universe changed, \Delta S_{\text{universe}} = 5.76n J/K. The process is irreversible, and the entropy increase is exactly the measure of how irreversible.
This also settles the intuitive question. Why does the gas never spontaneously gather itself back into one half? Because that would decrease entropy, and the counting argument below turns that into a probability you can actually write down.
Heat flowing across a temperature difference
Put a hot body at T_H in contact with a cold one at T_C and let a small amount of heat Q flow. Take both bodies large enough that their temperatures barely shift.
The hot body loses heat: \Delta S_H = -Q/T_H. The cold body gains it: \Delta S_C = +Q/T_C.
\Delta S_{\text{universe}} = Q\left(\frac{1}{T_C} - \frac{1}{T_H}\right)
Since T_C < T_H, the term 1/T_C is larger, so the bracket is positive and \Delta S > 0. Heat flowing hot to cold always creates entropy.
Now try running it the other way, with Q flowing from cold to hot. Every sign flips and \Delta S < 0, which the second law forbids. So Clausius's statement from Chapter 3.4 is not a separate axiom — it is a consequence of \Delta S \geq 0. One inequality now contains the whole law.
Worked number. 1000 J flows from a 500 K furnace to a 300 K room:
\Delta S = 1000\left(\frac{1}{300} - \frac{1}{500}\right) = 1000(0.003333 - 0.002000) = 1.333\ \text{J/K}
And notice the loss. A Carnot engine between those temperatures could have extracted W = 1000(1 - 300/500) = 400 J of work. By letting the heat flow directly instead, that 400 J is gone forever. The relationship is exact:
W_{\text{lost}} = T_C\,\Delta S_{\text{universe}} = 300\times1.333 = 400\ \text{J}
Entropy generated is work destroyed, at an exchange rate of the ambient temperature. That single sentence is why engineers care about entropy at all: every joule per kelvin you create is 300 joules of work you will never have.
Definition two: counting
Everything so far treated entropy as a bookkeeping device with no picture attached. Boltzmann supplied the picture, and it cost him dearly — atoms were still controversial in the 1890s and his work was attacked for resting on them.
Start with two words that must be kept apart.
A macrostate is what you can measure from outside: pressure, temperature, volume. A microstate is one complete specification of every molecule's position and velocity. An enormous number of microstates look identical from outside.
The number of microstates belonging to a given macrostate is called W, from the German Wahrscheinlichkeit, probability. Boltzmann's insight:
\boxed{S = k_B \ln W}
Read aloud: S equals k-B times the natural log of W. Entropy is the Boltzmann constant times the logarithm of the number of ways.
Boltzmann died by suicide in 1906, in poor health and worn down by decades of opposition to the atomic picture. Within a few years Einstein's analysis of Brownian motion and Perrin's experiments had settled the matter entirely in his favour.
Why a logarithm? Because entropy has to add up when you put two systems side by side, and the number of ways multiplies. If box A has W_A arrangements and box B has W_B, the pair has W_AW_B. The logarithm is the function that turns multiplication into addition:
S = k_B\ln(W_AW_B) = k_B\ln W_A + k_B\ln W_B = S_A + S_B
So the log is not a choice; it is forced by the requirement that entropy be additive. Volume I, Chapter 1.8 makes exactly the same argument for Shannon's information entropy, and gets a logarithm for exactly the same reason.
Checking the two definitions against each other
Take the free expansion again and compute it by counting. Before puncturing, each molecule can be anywhere in volume V. After, anywhere in 2V. So each molecule has twice as many places to be, and for N molecules the number of arrangements is multiplied by 2^N:
\Delta S = k_B\ln(2^NW_{\text{initial}}) - k_B\ln W_{\text{initial}} = k_B\ln(2^N) = Nk_B\ln 2
And Nk_B = nR, so:
\Delta S = nR\ln 2
Identical to the thermodynamic answer. One definition came from heat engines and thermometers with no atoms in sight; the other came from counting arrangements of molecules with no heat in sight. They agree exactly, which is the strongest evidence available that both are describing something real.
Why the gas never goes back
Now the probability question can be answered numerically. What are the odds that all N molecules happen to be in the left half at some instant? Each molecule is in the left half with probability 1/2, independently, so:
P = \left(\frac{1}{2}\right)^N
For one mole, N = 6\times10^{23}:
P = 2^{-6\times10^{23}} = 10^{-1.8\times10^{23}}
That exponent has 23 digits in it. To feel the scale: there are about 10^{80} atoms in the observable universe and about 10^{18} seconds since the Big Bang. If every atom in the universe checked once per second for the entire age of the universe, you would have made 10^{98} checks, which is not remotely a dent in 10^{1.8\times10^{23}}.
So the second law is not absolute in the way conservation of energy is. It is statistical. Nothing forbids the gas from gathering in one half; it is merely so overwhelmingly outnumbered that it will not happen in any universe that has ever existed. With four molecules instead of 10^{23}, the probability is 1/16 and you would see it constantly — and indeed for very small systems, entropy really does decrease for short intervals, which has been measured directly in experiments on single colloidal beads.
The arrow of time
Here is the puzzle that took a century to state properly. Every fundamental law of motion is symmetric in time. Film two billiard balls colliding, run the film backwards, and what you see obeys Newton's laws perfectly — you cannot tell which direction is forward. The same holds for Maxwell's equations, for general relativity, and very nearly for quantum mechanics.
So where does the obvious one-wayness of the world come from, if the underlying rules do not have it?
The answer has two halves.
First half: the counting. Run the film backwards and every individual collision is legal. What is not legal is the statistics. The reversed film shows a scrambled egg reassembling, which is not a forbidden sequence of collisions — it is a sequence that requires the initial velocities of 10^{23} molecules to be fine-tuned to a precision no process could produce. Forward in time, almost every microstate leads to a higher-entropy macrostate simply because higher-entropy macrostates contain almost all the microstates.
Second half, and it is the harder one: the past. The counting argument is symmetric. It says the future is probably higher-entropy, and by exactly the same logic it says the past was probably higher-entropy too, which is flatly wrong — the past was lower. So the argument alone cannot produce an arrow. Something extra is needed, and the only candidate anyone has found is a fact about initial conditions: the universe began in an extraordinarily low-entropy state.
Everything that has happened since is the universe rolling downhill from that start. Every fire, every star, every living thing, and every thought you have ever had is powered by the gap between the entropy the universe has and the entropy it will eventually reach. Chapter 12.8 asks what happens when the gap closes.
This makes the psychological arrow of time a thermodynamic one too. Forming a memory means recording information, and recording information in a physical medium always dumps waste heat into the surroundings. You remember the past and not the future because memory formation is itself an entropy-increasing process, and it can only run in the direction entropy runs.
The bridge to information
Volume I, Chapter 1.8 defined Shannon's information entropy for a set of possible messages:
H = -\sum_i p_i\log_2 p_i
If all W possibilities are equally likely, each has p_i = 1/W and this collapses to H = \log_2 W bits.
Boltzmann's is S = k_B\ln W joules per kelvin.
These are the same quantity in different units. Converting between \log_2 and \ln costs a factor of \ln 2, so:
S = k_B\ln 2 \times H
One bit of information corresponds to k_B\ln 2 = 9.57\times10^{-24} J/K of thermodynamic entropy. Shannon arrived at his formula in 1948 working on telephone lines, with no thermodynamics in mind; the story goes that von Neumann suggested he call it entropy partly because the formula matched Boltzmann's and partly because "nobody knows what entropy really is, so in a debate you will always have the advantage".
The connection is not a pun, and the proof of that is Landauer's principle, established by Rolf Landauer at IBM in 1961. Erasing one bit of information — genuinely erasing it, so that two possible prior states become one — must dissipate at least k_BT\ln 2 of energy as heat. At room temperature that is 2.9\times10^{-21} J, or about 0.018 electron-volts.
The reasoning: before erasure the bit could be 0 or 1, so there are two microstates. After, there is one. The information-bearing system's entropy fell by k_B\ln 2, and since the total cannot fall, at least that much must have appeared in the surroundings as heat.
This was confirmed experimentally in 2012 with a single colloidal particle in an optical trap, and the measured heat matched the prediction. It also settles Maxwell's demon, a puzzle Maxwell posed in 1867: imagine a tiny being operating a frictionless trapdoor between two gas chambers, letting fast molecules through one way and slow ones the other, thereby creating a temperature difference from nothing and breaking the second law. The resolution, worked out by Szilard, Landauer and Bennett across a century, is that the demon must measure each molecule and store the result, and its memory is finite, so eventually it must erase — and the erasure costs at least as much entropy as the sorting gained. The demon does not break the second law; it obeys it in its own memory.
And the practical consequence: Landauer's limit is the floor on the energy cost of irreversible computation. A modern processor dissipates thousands of times more than k_BT\ln 2 per logic operation, so there is enormous headroom left, but the floor is real and it is thermodynamic, not technological.
The plot puts every calculation in this chapter into one picture. The smooth climbing sections are mc\ln(T_2/T_1). The vertical jumps are mL/T. And the fact that the curve starts near zero at the left edge rather than diving to negative infinity is the third law, which Chapter 3.6 takes up.
Where this shows up in your life
Your body is an entropy exporter. A human maintains an exquisitely ordered internal structure, which is a local entropy decrease, and pays for it by taking in low-entropy energy (food, chemically ordered) and expelling high-entropy waste (heat at body temperature, carbon dioxide, water). A resting adult radiates about 100 W of heat at 310 K, which exports roughly 100/310 = 0.32 J/K every second. Life does not violate the second law; life is a machine for obeying it faster than the surroundings would alone.
Data centres are heat engines with no work output. Every watt of electricity becomes waste heat, and the cooling bill is the second law presenting its invoice. Some operators now pipe that heat into district heating systems, which is the only honest way to recover it.
Mixing is free and unmixing is not. Stir milk into coffee and it takes a second; extracting the milk again would take a laboratory. That asymmetry has a number attached — the entropy of mixing — and it is why separation processes (desalination, distillation, recycling mixed plastics) are always the expensive step in any industry.
Why you cannot un-scramble an egg, in one line: the scrambled state has vastly more microstates than the ordered one, and getting back requires steering 10^{23} molecules to specific places, which is exactly what a low-entropy configuration means and exactly what costs work.
What the next chapter fixes
The entropy curve above starts somewhere definite at the left edge, and nothing so far says where. Chapter 3.6 supplies the third law, which fixes that starting point at zero and shows why absolute zero can be approached forever but never reached. It also drops the ideal-gas assumptions from Chapter 3.2 — no molecular volume, no attractions — and finds out what happens when they are put back: gases that can condense, a critical point beyond which liquid and gas become indistinguishable, and the phase diagram that explains why ice skates work and why carbon dioxide has no liquid form at ordinary pressure.