Thermodynamics & Statistical Mechanics — Entropy, Free Energy, and the Boltzmann Distribution

Thermodynamics asks what energy transformations are possible; statistical mechanics explains why from the statistics of microscopic constituents. Together they describe steam engines, biochemical reactions, black holes, and the ultimate fate of the universe. This guide covers the four laws, Carnot efficiency, entropy, free energies, the Boltzmann distribution, ensembles, and phase transitions.

The Four Laws of Thermodynamics

Zeroth Law: If A is in thermal equilibrium with B, and B with C, then A is in thermal equilibrium with C. This defines temperature as an equivalence class and justifies thermometers.

First Law (energy conservation): dU = δQ − δW. The internal energy U of a closed system changes by heat added Q minus work done W. For a reversible process: dU = TdS − pdV (where S is entropy, T temperature, p pressure, V volume).

Second Law: The total entropy of an isolated system never decreases: dS ≥ 0. For irreversible processes dS > 0; for reversible processes dS = 0. Entropy quantifies the “irreversibility” or “spread of energy” in a system.

Third Law (Nernst theorem): As T → 0 K, the entropy of a perfect crystal approaches zero: S → 0. This makes absolute entropy well-defined and sets the lower limit on temperature.

Entropy & the Arrow of Time

The Boltzmann formula S = kB ln W connects macroscopic entropy to W, the number of accessible microstates for a given macrostate. Here kB = 1.38 × 10−²³ J/K is Boltzmann’s constant. A glass of water has ∼10²&sup4; molecules; the number of microstates consistent with “liquid water at 20 °C” is astronomically larger than for “all molecules in one corner”, so the latter is never observed spontaneously.

The arrow of time emerges from the second law: the past has lower entropy than the future. The fundamental microscopic laws (Newton, Schrödinger) are time-reversible, yet macroscopic evolution is overwhelmingly one-directional. The resolution is probabilistic: the initial state of the universe had extremely low entropy (the past hypothesis), and the second law holds statistically, not deterministically. Very rarely (in practice, never observably), entropy could decrease spontaneously — this is the content of the fluctuation theorem.

🌡️

Carnot Cycle Simulator lets you run a reversible heat engine and observe that the Carnot efficiency η = 1 − TC/TH is an upper bound for any heat engine operating between TH and TC. Try lowering TC toward 0 K to see η → 100%.

Carnot Cycle & Heat Engines

The Carnot cycle is the most efficient possible heat engine cycle between two thermal reservoirs at TH and TC < TH. It consists of four reversible strokes: isothermal expansion (heat absorbed QH at TH); adiabatic expansion (temperature drops to TC); isothermal compression (heat rejected QC at TC); adiabatic compression (temperature rises back to TH).

Carnot efficiency: ηC = 1 − TC/TH = W/QH
Entropy change: ΔS = QH/TH − QC/TC = 0 (reversible)
COP (refrigerator): COP = QC/W = TC/(TH−TC)

No engine can exceed Carnot efficiency (Kelvin-Planck statement of the second law). A coal plant (TH ≈ 800 K, TC ≈ 300 K): ηC ≈ 63%, actual ≈ 40%. A car engine (TH ≈ 1000 K): ηC ≈ 70%, actual ≈ 25–35% (irreversibilities in combustion, friction, throttling).

Thermodynamic Potentials & Free Energy

Different experimental constraints (fixed T, p, V, or N) require different thermodynamic potentials as the natural variables:

Internal energy: U(S,V)     dU = TdS − pdV
Helmholtz free energy: A = U − TS   dA = −SdT − pdV
Enthalpy: H = U + pV     dH = TdS + Vdp
Gibbs free energy: G = H − TS   dG = −SdT + Vdp

Spontaneous at const T,p: ΔG ≤ 0
Spontaneous at const T,V: ΔA ≤ 0

The Gibbs free energy G is the workhorse of chemistry: a reaction at constant T and p is spontaneous iff ΔG < 0 (which combines both enthalpy ΔH and entropy ΔS: ΔG = ΔH − TΔS). At equilibrium ΔG = 0. The Maxwell relations follow from the equality of mixed second derivatives: (∂T/∂V)S = −(∂p/∂S)V, etc. — connecting experimentally measurable quantities like the thermal expansion coefficient and isothermal compressibility.

The Boltzmann Distribution & Partition Function

Consider a system in thermal contact with a heat reservoir at temperature T. The probability of finding the system in microstate i with energy Ei is the Boltzmann distribution:

Pi = exp(−Ei/kBT) / Z

Partition function: Z = ∑i exp(−Ei/kBT)
(canonical ensemble, fixed N, V, T)

Free energy: A = −kBT ln Z
Mean energy: <E> = −∂ ln Z/∂β   (β = 1/kBT)
Heat capacity: CV = kBβ² ∂² ln Z/∂β²

The partition function Z encodes all thermodynamic information. Once Z is known (by summing over all microstates), every observable follows by differentiation. For N ideal gas particles in box volume V: Z1 = V (2πmkBT/h²)3/2 (de Broglie thermal wavelength). This gives the ideal gas law pV = NkBT and Sackur-Tetrode entropy directly from statistical mechanics, without phenomenological assumptions.

Examples

Statistical Ensembles

Different physical constraints define different ensembles:

In the thermodynamic limit (N, V → ∞, N/V fixed) all ensembles give the same macroscopic predictions. Differences appear only for small systems (nanoparticles, molecules) where fluctuations (relative SD ∼ 1/√N) are measurable.

Phase Transitions & Critical Phenomena

A first-order transition (liquid–gas, solid–liquid) involves a discontinuity in the first derivative of G: latent heat L = TΔS ≠ 0. The two phases coexist along the coexistence curve (Clausius-Clapeyron: dp/dT = L/(TΔV)). At the critical point (water: Tc = 374 °C, pc = 221 bar) the distinction between liquid and gas vanishes; the correlation length ξ diverges.

A second-order (continuous) transition has no latent heat but shows diverging fluctuations. The Ising model (spins ±1 on a lattice, coupled by J) is the canonical model. Mean-field theory predicts critical temperature kBTc = zJ (z = coordination number) and critical exponent β = 1/2 (magnetisation M ∝ |T−Tc|β). Exact 2D solution (Onsager 1944): β = 1/8. The renormalisation group (Wilson 1971, Nobel 1982) explains universality: systems with different microscopic details share the same critical exponents if they have the same symmetry and dimensionality.

Order parameter M ∝ (Tc−T)β   (T < Tc)
Specific heat: C ∝ |T−Tc|−α
Susceptibility: χ ∝ |T−Tc|−γ
Correlation length: ξ ∝ |T−Tc|−ν

Scaling relation: α + 2β + γ = 2 (Rushbrooke)
3D Ising exponents: α≈0.110, β≈0.326, γ≈1.237, ν≈0.630

Quantum Statistics

Identical quantum particles are fundamentally indistinguishable. Symmetry of the wavefunction under particle exchange defines two classes:

Fermi energy (3D): EF = (ℏ²/2m)(3π²n)²⁄³
Stefan-Boltzmann: P = σT&sup4;   (σ = 5.67×10−8 W/(m²K&sup4;))
Planck blackbody: u(ν,T) = (8πhν³/c³)/(ehν/kBT−1)
Wien displacement: λmaxT = 2.898 × 10−³ m·K

Thermodynamics & Information Theory

Claude Shannon (1948) defined information entropy H = −∑ pi log2 pi bits, formally identical to thermodynamic entropy S = −kB ∑ pi ln pi. This is not coincidental: information is physical.

Maxwell’s demon (1867) thought experiment: an intelligent being sorts fast/slow molecules between two chambers, apparently decreasing entropy without doing work, violating the second law. Resolution (Szilárd 1929, Landauer 1961): Landauer’s principle states that erasing one bit of information requires at least kBT ln 2 of energy dissipated as heat. The demon’s memory must be reset each cycle (or grows without bound), consuming at least as much entropy as it removed. Information erasure is physically irreversible.

Applications: DNA replication and cellular metabolism obey Landauer bounds; quantum computation (reversible gates) dissipates zero energy in principle; black hole evaporation (Hawking temperature TH = ℏc³/(8πGM kB)) connects gravity, thermodynamics, and quantum information via the Bekenstein-Hawking entropy S = A/(4 lP²) (area in Planck units).

🔥

Explore Carnot Cycle, Blackbody Radiation (Planck distribution, Wien peak, Stefan-Boltzmann), and Phase Equilibrium (Gibbs free energy, Le Chatelier’s principle) to see statistical mechanics at work.