210_COURSE THERMO UC SAN DIEGO

1. Lecture Notes on Thermodynamics and Statistical Mechanics (A Work in Progress) Daniel Arovas Department of Physics University of California, San Diego November 14, 2013

2. Contents 0.1 Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii 0.2 General references . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv 1 Probability 1 1.1 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 A Statistical View . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2.1 Distributions for a random walk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2.2 Thermodynamic limit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.2.3 Entropy and energy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.2.4 Entropy and information theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.3 Probability Distributions from Maximum Entropy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.3.1 The principle of maximum entropy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.3.2 Continuous probability distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 1.4 General Aspects of Probability Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1.4.1 Discrete and continuous distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1.4.2 Central limit theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.4.3 Multidimensional Gaussian integral . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 1.5 Appendix : Bayesian Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 2 Thermodynamics 17 2.1 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.2 What is Thermodynamics? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.2.1 Thermodynamic systems and state variables . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.2.2 Heat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 i

3. ii CONTENTS 2.2.3 Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.2.4 Pressure and Temperature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.2.5 Standard temperature and pressure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 2.3 The Zeroth Law of Thermodynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 2.4 Mathematical Interlude : Exact and Inexact Differentials . . . . . . . . . . . . . . . . . . . . . . . . . 24 2.5 The First Law of Thermodynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 2.5.1 Conservation of energy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 2.5.2 Single component systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 2.5.3 Ideal gases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 2.5.4 Adiabatic transformations of ideal gases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 2.5.5 Adiabatic free expansion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 2.6 Heat Engines and the Second Law of Thermodynamics . . . . . . . . . . . . . . . . . . . . . . . . . . 33 2.6.1 There’s no free lunch so quit asking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 2.6.2 Engines and refrigerators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 2.6.3 Nothing beats a Carnot engine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 2.6.4 The Carnot cycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 2.6.5 The Stirling cycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 2.6.6 The Otto and Diesel cycles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 2.6.7 The Joule-Brayton cycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 2.6.8 Carnot engine at maximum power output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 2.7 The Entropy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 2.7.1 Entropy and heat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 2.7.2 The Third Law of Thermodynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 2.7.3 Entropy changes in cyclic processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 2.7.4 Gibbs-Duhem relation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 2.7.5 Entropy for an ideal gas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 2.7.6 Example system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 2.7.7 Measuring the entropy of a substance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 2.8 Thermodynamic Potentials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 Energy E . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.8.1 52 Helmholtz free energy F . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.8.2 52

4. CONTENTS iii 2.8.3 Enthalpy H . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 Gibbs free energy G . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.8.4 54 Grand potential Ω . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.8.5 55 2.9 Maxwell Relations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 Relations deriving from E(S,V,N) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.9.1 55 Relations deriving from F(T,V,N) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.9.2 56 Relations deriving from H(S,p,N) 2.9.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 Relations deriving from G(T,p,N) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.9.4 56 Relations deriving from Ω(T,V,µ) 2.9.5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 2.9.6 Generalized thermodynamic potentials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 2.10 Equilibrium and Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 2.11 Applications of Thermodynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 2.11.1 Adiabatic free expansion revisited . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 2.11.2 Energy and volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 2.11.3 van der Waals equation of state . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 2.11.4 Thermodynamic response functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 2.11.5 Joule effect: free expansion of a gas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 2.11.6 Throttling: the Joule-Thompson effect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 2.12 Phase Transitions and Phase Equilibria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 p-v-T surfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.12.1 70 2.12.2 The Clausius-Clapeyron relation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 Liquid-solid line in H2O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.12.3 73 2.12.4 Slow melting of ice : a quasistatic but irreversible process . . . . . . . . . . . . . . . . . . . . 75 2.12.5 Gibbs phase rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 2.13 Entropy of Mixing and the Gibbs Paradox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 2.13.1 Computing the entropy of mixing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 2.13.2 Entropy and combinatorics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 2.13.3 Weak solutions and osmotic pressure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 2.13.4 Effect of impurities on boiling and freezing points . . . . . . . . . . . . . . . . . . . . . . . . 83 2.13.5 Binary solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 2.14 Some Concepts in Thermochemistry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

5. iv CONTENTS 2.14.1 Chemical reactions and the law of mass action . . . . . . . . . . . . . . . . . . . . . . . . . . 92 2.14.2 Enthalpy of formation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 2.14.3 Bond enthalpies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 2.15 Appendix I : Integrating factors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 2.16 Appendix II : Legendre Transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 2.17 Appendix III : Useful Mathematical Relations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 3 Ergodicity and the Approach to Equilibrium 107 3.1 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 3.2 Modeling the Approach to Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 3.2.1 Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 3.2.2 The Master Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 3.2.3 Equilibrium distribution and detailed balance . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 3.2.4 Boltzmann’s H-theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 3.3 Phase Flows in Classical Mechanics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 3.3.1 Hamiltonian evolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 3.3.2 Dynamical systems and the evolution of phase space volumes . . . . . . . . . . . . . . . . . 111 3.3.3 Liouville’s equation and the microcanonical distribution . . . . . . . . . . . . . . . . . . . . . 114 3.4 Irreversibility and Poincar´ e Recurrence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 3.4.1 Poincar´ e recurrence theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 3.4.2 Kac ring model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 3.5 Remarks on Ergodic Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 3.5.1 Definition of ergodicity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 3.5.2 The microcanonical ensemble . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 3.5.3 Ergodicity and mixing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 3.6 Thermalization of Quantum Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 3.6.1 Quantum dephasing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 3.6.2 Eigenstate thermalization hypothesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 3.6.3 When is the ETH true? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 3.7 Appendix I : Formal Solution of the Master Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 3.8 Appendix II : Radioactive Decay . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130

6. CONTENTS v 3.9 Appendix III : Canonical Transformations in Hamiltonian Mechanics . . . . . . . . . . . . . . . . . . 131 4 Statistical Ensembles 133 4.1 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 Microcanonical Ensemble (µCE) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 4.2 4.2.1 The microcanonical distribution function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 4.2.2 Density of states . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 Arbitrariness in the definition of S(E) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 4.2.3 4.2.4 Ultra-relativistic ideal gas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 4.2.5 Discrete systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 4.3 The Quantum Mechanical Trace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 4.3.1 The density matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 4.3.2 Averaging the DOS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 4.3.3 Coherent states . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 4.4 Thermal Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 4.5 Ordinary Canonical Ensemble (OCE) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 4.5.1 Canonical distribution and partition function . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 The difference between P(En) and Pn 4.5.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145 4.5.3 Averages within the OCE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145 4.5.4 Entropy and free energy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 4.5.5 Fluctuations in the OCE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 4.5.6 Thermodynamics revisited . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148 4.5.7 Generalized susceptibilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 4.6 Grand Canonical Ensemble (GCE) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150 4.6.1 Grand canonical distribution and partition function . . . . . . . . . . . . . . . . . . . . . . . 150 4.6.2 Entropy and Gibbs-Duhem relation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 4.6.3 Generalized susceptibilities in the GCE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 4.6.4 Fluctuations in the GCE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 4.6.5 Gibbs ensemble . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 4.7 Statistical Ensembles from Maximum Entropy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 µCE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 4.7.1

7. vi CONTENTS 4.7.2 OCE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 4.7.3 GCE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 4.8 Ideal Gas Statistical Mechanics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 4.8.1 Maxwell velocity distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 4.8.2 Equipartition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158 4.8.3 Quantum statistics and the Maxwell-Boltzmann limit . . . . . . . . . . . . . . . . . . . . . . 159 4.9 Selected Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160 4.9.1 Spins in an external magnetic field . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160 4.9.2 Negative temperature (!) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 4.9.3 Adsorption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 4.9.4 Elasticity of wool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164 4.9.5 Noninteracting spin dimers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 4.10 Statistical Mechanics of Molecular Gases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167 4.10.1 Separation of translational and internal degrees of freedom . . . . . . . . . . . . . . . . . . . 167 4.10.2 Ideal gas law . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 4.10.3 The internal coordinate partition function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 4.10.4 Rotations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 4.10.5 Vibrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 4.10.6 Two-level systems : Schottky anomaly . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172 4.10.7 Electronic and nuclear excitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174 4.11 Appendix I : Additional Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176 4.11.1 Three state system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176 4.11.2 Spins and vacancies on a surface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176 4.11.3 Fluctuating interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178 4.11.4 Dissociation of molecular hydrogen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180 5 Noninteracting Quantum Systems 183 5.1 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 5.2 Statistical Mechanics of Noninteracting Quantum Systems . . . . . . . . . . . . . . . . . . . . . . . . 184 5.2.1 Bose and Fermi systems in the grand canonical ensemble . . . . . . . . . . . . . . . . . . . . 184 5.2.2 Maxwell-Boltzmann limit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185

8. CONTENTS vii 5.2.3 Single particle density of states . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186 5.3 Quantum Ideal Gases : Low Density Expansions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187 5.3.1 Expansion in powers of the fugacity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187 5.3.2 Virial expansion of the equation of state . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187 5.3.3 Ballistic dispersion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189 5.4 Entropy and Counting States . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190 5.5 Photon Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191 5.5.1 Thermodynamics of the photon gas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191 5.5.2 Classical arguments for the photon gas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 5.5.3 Surface temperature of the earth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194 5.5.4 Distribution of blackbody radiation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 5.5.5 What if the sun emitted ferromagnetic spin waves? . . . . . . . . . . . . . . . . . . . . . . . . 196 5.6 Lattice Vibrations : Einstein and Debye Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196 5.6.1 One-dimensional chain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196 5.6.2 General theory of lattice vibrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 5.6.3 Einstein and Debye models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201 5.6.4 Melting and the Lindemann criterion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203 5.6.5 Goldstone bosons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206 5.7 The Ideal Bose Gas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207 5.7.1 General formulation for noninteracting systems . . . . . . . . . . . . . . . . . . . . . . . . . . 207 5.7.2 Ballistic dispersion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208 5.7.3 Isotherms for the ideal Bose gas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212 The λ-transition in Liquid4He . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213 5.7.4 Fountain effect in superfluid4He . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215 5.7.5 5.7.6 Bose condensation in optical traps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216 5.7.7 Example problem from Fall 2004 UCSD graduate written exam . . . . . . . . . . . . . . . . . 218 5.8 The Ideal Fermi Gas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219 5.8.1 Grand potential and particle number . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219 5.8.2 The Fermi distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220 T = 0 and the Fermi surface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220 5.8.3 5.8.4 Spin-split Fermi surfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222

9. viii CONTENTS 5.8.5 The Sommerfeld expansion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223 5.8.6 Chemical potential shift . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225 5.8.7 Specific heat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226 5.8.8 Magnetic susceptibility and Pauli paramagnetism . . . . . . . . . . . . . . . . . . . . . . . . 226 5.8.9 Landau diamagnetism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228 5.8.10 White dwarf stars . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230 6 Classical Interacting Systems 233 6.1 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233 6.2 Ising Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234 6.2.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234 6.2.2 Ising model in one dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234 H = 0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235 6.2.3 6.2.4 Chain with free ends . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236 6.2.5 Ising model in two dimensions : Peierls’ argument . . . . . . . . . . . . . . . . . . . . . . . . 237 6.2.6 Two dimensions or one? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239 6.2.7 High temperature expansion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241 6.3 Nonideal Classical Gases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243 6.3.1 The configuration integral . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244 6.3.2 One-dimensional Tonks gas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244 6.3.3 Mayer cluster expansion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245 6.3.4 Cookbook recipe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250 6.3.5 Lowest order expansion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251 6.3.6 Hard sphere gas in three dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252 6.3.7 Weakly attractive tail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253 6.3.8 Spherical potential well . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253 6.3.9 Hard spheres with a hard wall . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255 6.4 Lee-Yang Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257 6.4.1 Analytic properties of the partition function . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257 6.4.2 Electrostatic analogy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258 6.4.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259

10. CONTENTS ix 6.5 Liquid State Physics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260 6.5.1 The many-particle distribution function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260 6.5.2 Averages over the distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261 6.5.3 Virial equation of state . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265 6.5.4 Correlations and scattering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267 6.5.5 Correlation and response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269 6.5.6 BBGKY hierarchy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271 6.5.7 Ornstein-Zernike theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272 6.5.8 Percus-Yevick equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273 6.5.9 Ornstein-Zernike approximation at long wavelengths . . . . . . . . . . . . . . . . . . . . . . 274 6.6 Coulomb Systems : Plasmas and the Electron Gas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276 6.6.1 Electrostatic potential . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276 6.6.2 Debye-H¨ uckel theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277 6.6.3 The electron gas : Thomas-Fermi screening . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279 6.7 Polymers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281 6.7.1 Basic concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281 6.7.2 Polymers as random walks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283 6.7.3 Flory theory of self-avoiding walks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285 6.7.4 Polymers and solvents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287 6.8 Appendix I : Potts Model in One Dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288 6.8.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288 6.8.2 Transfer matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288 6.9 Appendix II : One-Particle Irreducible Clusters and the Virial Expansion . . . . . . . . . . . . . . . . 291 6.9.1 Irreducible clusters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292 7 Mean Field Theory of Phase Transitions 295 7.1 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295 7.2 The van der Waals system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296 7.2.1 Equation of state . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296 7.2.2 Analytic form of the coexistence curve near the critical point . . . . . . . . . . . . . . . . . . 299 7.2.3 History of the van der Waals equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302

11. x CONTENTS 7.3 Fluids, Magnets, and the Ising Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305 7.3.1 Lattice gas description of a fluid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305 7.3.2 Phase diagrams and critical exponents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306 7.3.3 Gibbs-Duhem relation for magnetic systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308 7.3.4 Order-disorder transitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308 7.4 Mean Field Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309 h = 0 7.4.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311 7.4.2 Specific heat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312 h 6= 0 Magnetization dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315 7.4.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313 7.4.4 7.4.5 Beyond nearest neighbors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317 7.4.6 Ising model with long-ranged forces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318 7.5 Variational Density Matrix Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319 7.5.1 The variational principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319 7.5.2 Variational density matrix for the Ising model . . . . . . . . . . . . . . . . . . . . . . . . . . . 320 7.5.3 Mean Field Theory of the Potts Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323 Mean Field Theory of the XY Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324 7.5.4 7.6 Landau Theory of Phase Transitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327 7.6.1 Cubic terms in Landau theory : first order transitions . . . . . . . . . . . . . . . . . . . . . . 329 7.6.2 Magnetization dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330 7.6.3 Sixth order Landau theory : tricritical point . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332 7.6.4 Hysteresis for the sextic potential . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333 7.7 Mean Field Theory of Fluctuations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336 7.7.1 Correlation and response in mean field theory . . . . . . . . . . . . . . . . . . . . . . . . . . . 336 7.7.2 Calculation of the response functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337 7.8 Global Symmetries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340 7.8.1 Symmetries and symmetry groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340 7.8.2 Lower critical dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341 7.8.3 Continuous symmetries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343 7.8.4 Random systems : Imry-Ma argument . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344 7.9 Ginzburg-Landau Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345

12. CONTENTS xi 7.9.1 Ginzburg-Landau free energy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345 7.9.2 Domain wall profile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346 7.9.3 Derivation of Ginzburg-Landau free energy . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347 7.9.4 Ginzburg criterion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351 7.10 Appendix I : Equivalence of the Mean Field Descriptions . . . . . . . . . . . . . . . . . . . . . . . . . 352 7.10.1 Variational Density Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353 7.10.2 Mean Field Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354 7.11 Appendix II : Additional Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355 7.11.1 Blume-Capel model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355 7.11.2 Ising antiferromagnet in an external field . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357 7.11.3 Canted quantum antiferromagnet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360 7.11.4 Coupled order parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362 8 Nonequilibrium Phenomena 367 8.1 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367 8.2 Equilibrium, Nonequilibrium and Local Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . . . . 368 8.3 Boltzmann Transport Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369 8.3.1 Derivation of the Boltzmann equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369 8.3.2 Collisionless Boltzmann equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371 8.3.3 Collisional invariants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372 8.3.4 Scattering processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373 8.3.5 Detailed balance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374 8.3.6 Kinematics and cross section . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376 8.3.7 H-theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376 8.4 Weakly Inhomogeneous Gas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 378 8.5 Relaxation Time Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 380 8.5.1 Approximation of collision integral . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 380 8.5.2 Computation of the scattering time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 380 8.5.3 Thermal conductivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381 8.5.4 Viscosity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382 8.5.5 Oscillating external force . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385

13. xii CONTENTS 8.5.6 Quick and Dirty Treatment of Transport . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386 8.5.7 Thermal diffusivity, kinematic viscosity, and Prandtl number . . . . . . . . . . . . . . . . . . 386 8.6 Diffusion and the Lorentz model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387 8.6.1 Failure of the relaxation time approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387 8.6.2 Modified Boltzmann equation and its solution . . . . . . . . . . . . . . . . . . . . . . . . . . 388 8.7 Linearized Boltzmann Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 390 8.7.1 Linearizing the collision integral . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 390 Linear algebraic properties ofˆL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 390 8.7.2 8.7.3 Steady state solution to the linearized Boltzmann equation . . . . . . . . . . . . . . . . . . . 392 8.7.4 Variational approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392 8.8 The Equations of Hydrodynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 396 8.9 Nonequilibrium Quantum Transport . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 396 8.9.1 Boltzmann equation for quantum systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 396 8.9.2 The Heat Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 400 8.9.3 Calculation of Transport Coefficients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401 8.9.4 Onsager Relations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402 8.10 Stochastic Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403 8.10.1 Langevin equation and Brownian motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403 8.10.2 Langevin equation for a particle in a harmonic well . . . . . . . . . . . . . . . . . . . . . . . 406 8.10.3 Discrete random walk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407 8.10.4 Fokker-Planck equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408 8.10.5 Brownian motion redux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409 8.10.6 Master Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 410 8.11 Appendix I : Boltzmann Equation and Collisional Invariants . . . . . . . . . . . . . . . . . . . . . . . 412 8.12 Appendix II : Distributions and Functionals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 414 8.13 Appendix III : General Linear Autonomous Inhomogeneous ODEs . . . . . . . . . . . . . . . . . . . 416 8.14 Appendix IV : Correlations in the Langevin formalism . . . . . . . . . . . . . . . . . . . . . . . . . . 422 8.15 Appendix V : Kramers-Kr¨ onig Relations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424

14. 0.1. PREFACE xiii 0.1 Preface This is a proto-preface. A more complete preface will be written after these notes are completed. These lecture notes are intended to supplement a course in statistical physics at the upper division undergraduate or beginning graduate level. I was fortunate to learn this subject from one of the great statistical physicists of our time, John Cardy. I am grateful to my wife Joyce and to my children Ezra and Lily for putting up with all the outrageous lies I’ve told them about getting off the computer ‘in just a few minutes’ while working on these notes. These notes are dedicated to the only two creatures I know who are never angry with me: my father and my dog. Figure 1: My father (Louis) and my dog (Henry).

15. xiv CONTENTS 0.2 General references – L. Peliti, Statistical Mechanics in a Nutshell (Princeton University Press, 2011) The best all-around book on the subject I’ve come across thus far. Appropriate for the graduate or advanced undergraduate level. – J. P. Sethna, Entropy, Order Parameters, and Complexity (Oxford, 2006) An excellent introductory text with a very modern set of topics and exercises. Available online at http://www.physics.cornell.edu/sethna/StatMech – M. Kardar, Statistical Physics of Particles (Cambridge, 2007) A superb modern text, with many insightful presentations of key concepts. – M. Plischke and B. Bergersen, Equilibrium Statistical Physics (3rdedition, World Scientific, 2006) An excellent graduatelevel text. Less insightful than Kardarbut still a good modern treatmentof the subject. Good discussion of mean field theory. – E. M. Lifshitz and L. P. Pitaevskii, Statistical Physics (part I, 3rdedition, Pergamon, 1980) This is volume 5 in the famous Landau and Lifshitz Course of Theoretical Physics. Though dated, it still contains a wealth of information and physical insight. – F. Reif, Fundamentals of Statistical and Thermal Physics (McGraw-Hill, 1987) This has been perhaps the most popular undergraduate text since it first appeared in 1967, and with good reason.

16. Chapter 1 Probability 1.1 References – F. Reif, Fundamentals of Statistical and Thermal Physics (McGraw-Hill, 1987) This has been perhaps the most popular undergraduate text since it first appeared in 1967, and with good reason. – E. T. Jaynes, Probability Theory (Cambridge, 2007) The bible on probability theory for physicists. A strongly Bayesian approach. – C. Gardiner, Stochastic Methods (Springer-Verlag, 2010) Very clear and complete text on stochastic mathematics. 1

17. 2 CHAPTER 1. PROBABILITY 1.2 A Statistical View 1.2.1 Distributions for a random walk Consider the mechanical system depicted in Fig. 1.1, a version of which is often sold in novelty shops. A ball is released from the top, which cascades consecutively through N levels. The details of each ball’s motion are governed by Newton’s laws of motion. However, to predict where any given ball will end up in the bottom row is difficult, because the ball’s trajectory depends sensitively on its initial conditions, and may even be influenced by random vibrations of the entire apparatus. We therefore abandon all hope of integrating the equations of motion and treat the system statistically. That is, we assume, at each level, that the ball moves to the right with probability p and to the left with probability q = 1− p. If there is no bias in the system, then p = q =1 N steps may be written N X where σj= +1 if the ball moves to the right at level j, and σj= −1 if the ball moves to the left at level j. At each level, the probability for these two outcomes is given by 2. The position XNafter X = σj, (1.1) j=1 ( p q if σ = +1 if σ = −1 . Pσ= pδσ,+1+ qδσ,−1= (1.2) This is a normalized discrete probability distribution of the type discussed in section 1.4 below. The multivariate distribution for all the steps is then N Y P(σ1, ... ,σN) = P(σj) . (1.3) j=1 Our system is equivalent to a one-dimensional random walk. Imagine an inebriated pedestrian on a sidewalk taking steps to the right and left at random. After N steps, the pedestrian’s location is X. Now let’s compute the average of X: N X X hXi =? ?= Nhσi = N σP(σ) = N(p − q) = N(2p − 1) . σj (1.4) σ=±1 j=1 This could be identified as an equation of state for our system, as it relates a measurable quantity X to the number of steps N and the local bias p. Next, let’s compute the average of X2: N N X X hX2i = hσjσj′i = N2(p − q)2+ 4Npq . (1.5) j′=1 j=1 Here we have used ( if j = j′ if j 6= j′. hσjσj′i = δjj′ +?1 − δjj′?(p − q)2= 1 (p − q)2 (1.6) Note that hX2i ≥ hXi2, which must be so because Var(X) = h(∆X)2i ≡??X − hXi?2?= hX2i − hXi2. pVar(X). Note that the mean value of X is linearly proportional to N1, but the RMS 2, where hXi = 0. (1.7) This is called the variance of X. We have Var(X) = 4Npq. The root mean square deviation, ∆Xrms, is the square root of the variance: ∆Xrms= 1The exception is the unbiased case p = q =1

18. 1.2. A STATISTICAL VIEW 3 Figure 1.1: The falling ball system, which mimics a one-dimensional random walk. fluctuations ∆Xrmsare proportional to N1/2. In the limit N → ∞ then, the ratio ∆Xrms/hXi vanishes as N−1/2. This is a consequence of the central limit theorem (see §1.4.2 below), and we shall meet up with it again on several occasions. We can do even better. We can find the complete probability distribution for X. It is given by ?N ? pNRqNL, PN,X= (1.8) NR where NR/Lare the numbers of steps taken to the right/left, with N = NR+ NL, and X = NR− NL. There are many independent ways to take NRsteps to the right. For example, our first NRsteps could all be to the right, and the remaining NL= N − NRsteps would then all be to the left. Or our final NRsteps could all be to the right. For each of these independent possibilities, the probability is pNRqNL. How many possibilities are there? Elementary combinatorics tells us this number is ?N Note that N ± X = 2NR/L, so we can replace NR/L=1 N! ?N+X ? N! = NR!NL!. (1.9) NR 2(N ± X). Thus, ?!p(N+X)/2q(N−X)/2. ?!?N−X PN,X= (1.10) 2 2 1.2.2 Thermodynamic limit Consider the limit N → ∞ but with x ≡ X/N finite. This is analogous to what is called the thermodynamic limit in statistical mechanics. Since N is large, x may be considered a continuous variable. We evaluate lnPN,Xusing Stirling’s asymptotic expansion lnN! ≃ N lnN − N + O(lnN) . (1.11)

19. 4 CHAPTER 1. PROBABILITY We then have h i lnPN,X≃ N lnN − N −1 −1 = −N 1 2N(1 + x) +1 2N(1 + x)ln 2N(1 + x) h i 2 1 2N(1 − x) ?ln?1+x +1 ?ln?1−x 2N(1 − x) +1 2N(1 + x) lnp +1 h?1+x 2N(1 − x)ln h?1+x 2N(1 − x) lnq ?lnq ?i i ?+?1−x ?lnp +?1−x + N . (1.12) 2 2 2 2 2 Notice that the terms proportional to N lnN have all cancelled, leaving us with a quantity which is linear in N. We may therefore write lnPN,X= −Nf(x) + O(lnN), where h?1+x ?i h?1+x i ?ln?1+x ?+?1−x ?ln?1−x ?lnp +?1−x ?lnq − f(x) = . (1.13) 2 2 2 2 2 2 We have just shown that in the large N limit we may write PN,X= C e−Nf(X/N), (1.14) where C is a normalization constant2. Since N is by assumption large, the function PN,Xis dominated by the minimum (orminima) of f(x), wherethe probabilityis maximized. To find the minimum of f(x), we set f′(x) = 0, where ?q Setting f′(x) = 0, we obtain 1 + x 1 − x=p We also have ? p·1 + x 1 − x f′(x) =1 2ln . (1.15) ⇒ ¯ x = p − q . (1.16) q 1 f′′(x) = 1 − x2, (1.17) so invoking Taylor’s theorem, 2f′′(¯ x)(x − ¯ x)2+ ... . f(x) = f(¯ x) +1 (1.18) Putting it all together, we have " # " # −(X −¯ X)2 8Npq −N(x − ¯ x)2 8pq PN,X≈ C exp = C exp , (1.19) where¯ X = hXi = N(p − q) = N¯ x. The constant C is determined by the normalization condition, Z −∞ and thus C = 1/√2πNpq. Why don’t we go beyond second order in the Taylor expansion of f(x)? We will find out in §1.4.2 below. " # ∞ ∞ X −(X −¯ X)2 8Npq p2πNpq C , PN,X≈1 dX C exp = (1.20) 2 X=−∞ 2The origin of C lies in the O(lnN) and O(N0) terms in the asymptotic expansion of lnN!. We have ignored these terms here. Accounting for them carefully reproduces the correct value of C in eqn. 1.20.

20. 1.2. A STATISTICAL VIEW 5 Figure 1.2: Comparison of exact distribution of eqn. 1.10 (red squares) with the Gaussian distribution of eqn. 1.19 (blue line). 1.2.3 Entropy and energy The function f(x) can be written as a sum of two contributions, f(x) = e(x) − s(x), where s(x) = −?1+x e(x) = −1 ?ln?1+x ?−?1−x ?ln?1−x ? 2 2 2xln(p/q) . 2 2 (1.21) 2ln(pq) −1 The function S(N,x) ≡ Ns(x) is analogous to the statistical entropy of our system3. We have ?N ? ? ? N S(N,x) = Ns(x) = ln = ln . (1.22) 1 2N(1 + x) NR Thus, the statistical entropy is the logarithm of the number of ways the system can be configured so as to yield the same value of X (at fixed N). The second contribution to f(x) is the energy term. We write E(N,x) = Ne(x) = −1 2N ln(pq) −1 2Nxln(p/q) . (1.23) The energy term biases the probability PN,X= exp(S − E) so that low energy configurations are more probable than high energy configurations. For our system, we see that when p < q (i.e. p <1 as small as possible (meaning as negative as possible). The smallest possible allowed value of x = X/N is x = −1. Conversely, when p > q (i.e. p >1 2), the energy is minimized by taking x as large as possible, which means x = 1. The average value of x, as we have computed explicitly, is ¯ x = p − q = 2p − 1, which falls somewhere in between these two extremes. 2), the energy is minimized by taking x In actual thermodynamic systems, as we shall see, entropy and energy are not dimensionless. What we have called S here is really S/kB, which is the entropy in units of Boltzmann’s constant. And what we have called E here is really E/kBT, which is energy in units of Boltzmann’s constant times temperature. 3The function s(x) is the specific entropy.

21. 6 CHAPTER 1. PROBABILITY 1.2.4 Entropy and information theory It was shown in the classic 1948work of Claude Shannon that entropy is in fact a measure of information4. Suppose we observe that a particular event occurs with probability p. We associate with this observation an amount of information I(p). The information I(p) should satisfy certain desiderata: 1 Information is non-negative, i.e. I(p) ≥ 0. 2 If two events occur independently so their joint probability is p1p2, then their information is additive, i.e. I(p1p2) = I(p1) + I(p2). 3 I(p) is a continuous function of p. 4 There is no information content to an event which is always observed, i.e. I(1) = 0. From these four properties, it is easy to show that the only possible function I(p) is I(p) = −A lnp , (1.24) where A is an arbitrary constant that can be absorbed into the base of the logarithm, since logbx = lnx/lnb. We will take A = 1 and use e as the base, so I(p) = −lnp. Another common choice is to take the base of the logarithm to be 2, so I(p) = −log2p. In this latter case, the units of information are known as bits. Note that I(0) = ∞. This means that the observation of an extremely rare event carries a great deal of information. Now suppose we have a set of events labeled by an integer n which occur with probabilities {pn}. What is the expected amount of information in N observations? Since event n occurs an average of Npntimes, and the information content in pnis −lnpn, we have that the average information per observation is S =hINi N X = − pnlnpn, (1.25) n which is known as the entropy of the distribution. Thus, maximizing S is equivalent to maximizing the information content per observation. Consider, for example, the information content of course grades. As we have seen, if the only constraint on the probability distribution is that of overall normalization, then S is maximized when all the probabilities pnare equal. The binary entropy is then S = log2Γ, since pn= 1/Γ. Thus, for pass/fail grading, the maximum average information per grade is −log2(1 maximum average information per grade is log25 = 2.32 bits. If we expand the grade options to include {A+, A, A-, B+, B, B-, C+, C, C-, D, F}, then the maximum average information per grade is log211 = 3.46 bits. Equivalently, consider, following the discussion in vol. 1 of Kardar, a random sequence {n1,n2,...,nN} where each element njtakes one of K possible values. There are then KNsuch possible sequences, and to specify one of them requires log2(KN) = N log2K bits of information. However, if the value n occurs with probability pn, then on average it will occur Nn= Npntimes in a sequence of length N, and the total number of such sequences will be g(N) = QK among these g(N) possibilities is 2) = log22 = 1 bit. If only A, B, C, D, and F grades are assigned, then the N! . (1.26) n=1Nn! In general, this is far less that the total possible number KN, and the number of bits necessary to specify one from K K X X log2g(N) = log2(N!) − log2(Nn!) ≈ −N pnlog2pn, (1.27) n=1 n=1 4See ‘An Introduction to Information Theory and Entropy’ by T. Carter, Santa Fe Complex Systems Summer School, June 2011. Available online at http://astarte.csustan.edu/ tom/SFI-CSSS/info-theory/info-lec.pdf.

22. 1.3. PROBABILITY DISTRIBUTIONS FROM MAXIMUM ENTROPY 7 1 Kfor all where we have invoked Stirling’s approximation. If the distribution is uniform, then we have pn= n ∈ {1,...,K}, and log2g(N) = N log2K. 1.3 Probability Distributions from Maximum Entropy We have shown how one can proceed from a probability distribution and compute various averages. We now seek to go in the other direction, and determine the full probability distribution based on a knowledge of certain averages. At first, this seems impossible. Suppose we want to reproduce the full probability distribution for an N-step random walk from knowledge of the average hXi = (2p − 1)N. The problem seems ridiculously underdeter- mined, since there are 2Npossible configurations for an N-step random walk: σj= ±1 for j = 1,...,N. Overall normalization requires X but this just imposes one constraint on the 2Nprobabilities P(σ1,...,σN), leaving 2N−1overall parameters. What principle allows us to reconstruct the full probability distribution P(σ1,...,σN) = 1 , (1.28) {σj} N Y N Y ?pδσj,1+ q δσj,−1 ?= p(1+σj)/2q(1−σj)/2, P(σ1,...,σN) = (1.29) j=1 j=1 corresponding to N independent steps? 1.3.1 The principle of maximum entropy The entropy of a discrete probability distribution {pn} is defined as X S = − pnlnpn, (1.30) n where here we take e as the base of the logarithm. The entropy may therefore be regarded as a function of the probability distribution: S = S?{pn}?. One special property of the entropy is the following. Suppose we have two The entropy of the joint distribution is then X = − a b b = SA+ SB. independent normalized distributions?pA S = − ?and?pB X X ?. The joint probability for events a and b is then Pa,b= pA pA b a b X apB b. a b X pA X pB X pA X a− bln?pA ?= − X ?lnpA pB ? apB apB pA apB a+ lnpB Pa,blnPa,b= − X b b a a b b X X alnpA pB blnpB pA alnpA blnpB a· b− b· a= − b a a b Thus, the entropy of a joint distribution formed from two independent distributions is additive. Suppose all we knew about {pn} was that it was normalized. ThenP constraint. We do this using Lagrange’s method of undetermined multipliers. We define S∗?{pn},λ?= − npn= 1. This is a constraint on the values {pn}. Let us now extremize the entropy S with respect to the distribution {pn}, but subject to the normalization ?X ? X pnlnpn− λ pn− 1 (1.31) n n

23. 8 CHAPTER 1. PROBABILITY and we freely extremize S∗over all its arguments. Thus, for all n we have ∂S∗ ∂pn = −?lnpn+ 1 + λ?= 0 X (1.32) as well as ∂S∗ ∂λ pn− 1 = 0 . = (1.33) n From the first of these equations, we obtain pn= e−(1+λ), and from the second we obtain X where Γ ≡P X pn= e−(1+λ)· 1 = Γ e−(1+λ), (1.34) n n n1 is the total number of possible events. Thus, pn=1 Γ, (1.35) which says that all events are equally probable. Now suppose we know one other piece of information, which is the average value of some quantity X = P S∗?{pn},λ0,λ1 We then have ∂S∗ ∂pn nXnpn. We now extremize S subject to two constraints, and so we define ?X ? ?X ? X ?= − pnlnpn− λ0 pn− 1 − λ1 Xnpn− X . (1.36) n n n = −?lnpn+ 1 + λ0+ λ1Xn ?= 0 , (1.37) which yields the two-parameter distribution pn= e−(1+λ0)e−λ1Xn. (1.38) To fully determine the distribution {pn} we need to invoke the two equationsP npn= 1 andP nXnpn= X, which come from extremizing S∗with respect to λ0and λ1, respectively: e−(1+λ0)X e−(1+λ0)X e−λ1Xn= 1 (1.39) n Xne−λ1Xn= X . (1.40) n General formulation The generalization to K extra pieces of information (plus normalization) is immediately apparent. We have X Xa= Xa npn, (1.41) n and therefore we define ?X npn− Xa? K X X S∗?{pn},{λa}?= − Xa pnlnpn− λa , (1.42) n n a=0

24. 1.3. PROBABILITY DISTRIBUTIONS FROM MAXIMUM ENTROPY 9 with X(a=0) ≡ X(a=0)= 1. Then the optimal distribution which extremizes S subject to the K + 1 constraints is ( a=0 ( a=1 P n ) K X K X λaXa − 1 − pn= exp n (1.43) ) =1 λaXa − Zexp , n where Z = e1+λ0is determined by normalization: {λ0,λ1,...,λK} determined by the K + 1 constraints in eqn. 1.41. npn= 1. This is a (K + 1)-parameter distribution, with Example As an example, consider the random walk problem. We have two pieces of information: X X X ··· P(σ1,...,σN) = 1 (1.44) σ1 σN N X X ··· P(σ1,...,σN) σj= X . (1.45) j=1 σ1 σN Here the discrete label n from §1.3.1 ranges over 2Npossible values, and may be written as an N digit binary number rN···r1, where rj=1 ( 2(1 + σj) is 0 or 1. Extremizing S subject to these constraints, we obtain ) N Y X e−λσj, P(σ1,...,σN) = C exp − λ = C σj (1.46) j j=1 where C ≡ e−(1+λ0)and λ ≡ λ2. Normalization then requires X = C?eλ+ e−λ?N, Tr P ≡ (1.47) {σj} hence C = (coshλ)−N. We then have N Y N Y e−λσj eλ+ e−λ= ?pδσj,1+ q δσj,−1 ?, P(σ1,...,σN) = (1.48) j=1 j=1 where e−λ eλ q = 1 − p = p = , eλ+ e−λ. (1.49) eλ+ e−λ We then have X = (2p − 1)N, which determines p =1 2(N + X), and we have recovered the correct distribution. 1.3.2 Continuous probability distributions Suppose we have a continuous probability density P(ϕ) defined over some set Ω. We have observables Z Ω Xa= dµ Xa(ϕ)P(ϕ) , (1.50)

25. 10 CHAPTER 1. PROBABILITY where dµ is the appropriate integration measure. We assume dµ =QD S∗?P(ϕ),{λa}?= − with respect to P(ϕ) and with respect to {λa}. Again, X0(ϕ) ≡ X0≡ 1. This yields the following result: j=1dϕj, where D is the dimension of Ω. Then we extremize the functional Z ! Z Ω K X dµ P(ϕ)Xa(ϕ) − Xa dµ P(ϕ)lnP(ϕ) − λa (1.51) a=0 Ω K X λaXa(ϕ) . lnP(ϕ) = −1 − (1.52) a=0 The K + 1 Lagrange multipliers {λa} are then determined from the K + 1 constraint equations in eqn. 1.50. As an example, consider a distribution P(x) over the real numbers R. We constrain ∞ Z −∞ ∞ Z −∞ ∞ Z −∞ dx x2P(x) = µ2+ σ2. dx P(x) = 1 , dx xP(x) = µ , (1.53) Extremizing the entropy, we then obtain P(x) = C e−λ1x−λ2x2, (1.54) where C = e−(1+λ0). We already know the answer: 1 √2πσ2e−(x−µ)2/2σ2. P(x) = (1.55) In other words, λ1= −µ/σ2and λ2= 1/2σ2, with C = e−µ2/2σ2/√2πσ2. 1.4 General Aspects of Probability Distributions 1.4.1 Discrete and continuous distributions Consider a system whose possible configurations |ni can be labeled by a discrete variable n ∈ C, where C is the set of possible configurations. The total number of possible configurations, which is to say the order of the set C, may be finite or infinite. Next, consider an ensemble of such systems, and let Pndenote the probability that a given random element from that ensemble is in the state (configuration) |ni. The collection {Pn} forms a discrete probability distribution. We assume that the distribution is normalized, meaning X Pn= 1 . (1.56) n∈C Now let Anbe a quantity which takes values depending on n. The average of A is given by X hAi = PnAn. (1.57) n∈C Typically, C is the set of integers (Z) or some subset thereof, but it could be any countable set. As an example, consider the throw of a single six-sided die. Then Pn=1 is odd. Then find hAi =1 6for each n ∈ {1,...,6}. Let An= 0 if n is even and 1 if n 2, i.e. on average half the throws of the die will result in an even number.

26. 1.4. GENERAL ASPECTS OF PROBABILITY DISTRIBUTIONS 11 It may be that the system’s configurations are described by several discrete variables {n1,n2,n3,...}. We can combine these into a vector n and then we write Pnfor the discrete distribution, withP Another possibility is that the system’s configurations are parameterized by a collection of continuous variables, ϕ = {ϕ1,...,ϕn}. We write ϕ ∈ Ω, where Ω is the phase space (or configuration space) of the system. Let dµ be a measure on this space. In general, we can write nPn= 1. dµ = W(ϕ1,...,ϕn)dϕ1dϕ2···dϕn. (1.58) The phase space measure used in classical statistical mechanics gives equal weight W to equal phase space vol- umes: rY where C is a constant we shall discuss later on below5. Any continuous probability distribution P(ϕ) is normalized according to Z Ω dµ = C dqσdpσ, (1.59) σ=1 dµP(ϕ) = 1 . (1.60) The average of a function A(ϕ) on configuration space is then Z Ω hAi = dµP(ϕ)A(ϕ) . (1.61) For example, consider the Gaussian distribution 1 √2πσ2e−(x−µ)2/2σ2. P(x) = (1.62) From the result6 rπ ∞ Z −∞ dx e−αx2e−βx= αeβ2/4α, (1.63) we see that P(x) is normalized. One can then compute hxi = µ (1.64) hx2i − hxi2= σ2. We call µ the mean and σ the standard deviation of the distribution, eqn. 1.62. The quantity P(ϕ) is called the distribution or probability density. One has P(ϕ)dµ = probability that configuration lies within volume dµ centered at ϕ For example, consider the probability density P = 1 normalized on the interval x ∈?0,1?. The probability that many digits of x. However, we can say that x ∈?0.45, 0.55?with probability and momenta which leave Hamilton’s equations of motion invariant, and which preserve phase space volumes under Hamiltonian evolution. For this reason dµ is called an invariant phase space measure. See the discussion in appendix II of chapter 4. 6Memorize this! some x chosen at random will be exactly1 2, say, is infinitesimal – one would have to specify each of the infinitely 1 10. 5Such a measure is invariant with respect to canonical transformations, which are the broad class of transformations among coordinates

27. 12 CHAPTER 1. PROBABILITY If x is distributed according to P1(x), then the probability distribution on the product space (x1, x2) is simply the product of the distributions: P2(x1,x2) = P1(x1)P1(x2) . (1.65) Suppose we have a function φ(x1,...,xN). How is it distributed? Let Q(φ) be the distribution for φ. We then have ∞ Z −∞ ∞ Z −∞ ∞ Z −∞ ∞ Z −∞ ? ? ? ? P(φ) = dx1··· φ(x1,...,xN) − φ dxNPN(x1,...,xN) δ (1.66) dx1··· xNP1(x1)···P1(xN) δ φ(x1,...,xN) − φ = , where the second line is appropriate if the {xj} are themselves distributed independently. Note that ∞ Z −∞ dφ P(φ) = 1 , (1.67) so P(φ) is itself normalized. 1.4.2 Central limit theorem In particular, consider the distribution function of the sum N X X = xi. (1.68) i=1 We will be particularly interested in the case where N is large. For general N, though, we have ∞ Z −∞ ∞ Z −∞ dxNP1(x1)···P1(xN) δ?x1+ x2+ ... + xN− X?. PN(X) = dx1··· (1.69)

28. 1.4. GENERAL ASPECTS OF PROBABILITY DISTRIBUTIONS 13 It is convenient to compute the Fourier transform7of P(X): ∞ Z −∞ ∞ Z −∞ ˆPN(k) = dX PN(X)e−ikX (1.70) ∞ Z −∞ ∞ Z −∞ xNP1(x1)···P1(xN) δ?x1+ ... + xN− X)e−ikX dx1··· = dX =?ˆP1(k)?N, where ∞ Z −∞ ˆP1(k) = dxP1(x)e−ikx (1.71) is the Fourier transform of the single variable distribution P1(x). The distribution PN(X) is a convolution of the individual P1(xi) distributions. We have therefore proven that the Fourier transform of a convolution is the product of the Fourier transforms. OK, now we can write forˆP1(k) ∞ Z −∞ dxP1(x)?1 − ikx −1 = 1 − ikhxi −1 6ik3x3+ ...? ˆP1(k) = 2k2x2+1 (1.72) 2k2hx2i +1 6ik3hx3i + ... . Thus, lnˆP1(k) = −iµk −1 2σ2k2+1 6iγ3k3+ ... , (1.73) where µ = hxi σ2= hx2i − hxi2 γ3= hx3i − 3hx2ihxi + 2hxi3 (1.74) We can now write ?ˆP1(k)?N= e−iNµke−Nσ2k2/2eiNγ3k3/6··· (1.75) 7Jean Baptiste Joseph Fourier (1768-1830) had an illustrious career. The son of a tailor, and orphaned at age eight, Fourier’s ignoble status rendered him ineligible to receive a commission in the scientific corps of the French army. A Benedictine minister at the´Ecole Royale Militaire of Auxerre remarked, ”Fourier, not being noble, could not enter the artillery, although he were a second Newton.” Fourier prepared for the priesthood, but his affinity for mathematics proved overwhelming, and so he left the abbey and soon thereafter accepted a military lectureship position. Despite his initial support for revolution in France, in 1794 Fourier ran afoul of a rival sect while on a trip to Orl´ eans and was arrested and very nearly guillotined. Fortunately the Reign of Terror ended soon after the death of Robespierre, and Fourier was released. He went on Napoleon Bonaparte’s 1798 expedition to Egypt, where he was appointed governor of Lower Egypt. His organizational skills impressed Napoleon, and upon return to France he was appointed to a position of prefect in Grenoble. It was in Grenoble that Fourier performed his landmark studies of heat, and his famous work on partial differential equations and Fourier series. It seems that Fourier’s fascination with heat began in Egypt, where he developed an appreciation of desert climate. His fascination developed into an obsession, and he became convinced that heat could promote a healthy body. He would cover himself in blankets, like a mummy, in his heated apartment, even during the middle of summer. On May 4, 1830, Fourier, so arrayed, tripped and fell down a flight of stairs. This aggravated a developing heart condition, which he refused to treat with anything other than more heat. Two weeks later, he died. Fourier’s is one of the 72 names of scientists, engineers and other luminaries which are engraved on the Eiffel Tower. Source: http://www.robertnowlan.com/pdfs/Fourier,%20Joseph.pdf

29. 14 CHAPTER 1. PROBABILITY Now for the inverse transform. In computing PN(X), we will expand the term eiNγ3k3/6and all subsequent terms in the above product as a power series in k. We then have ∞ Z −∞ ? √2πNσ2e−(X−Nµ)2/2Nσ2 dk 2πeik(X−Nµ)e−Nσ2k2/2n o 6iNγ3k3+ ... 1 +1 PN(X) = ? 6Nγ3∂3 1 (1.76) √2πNσ2e−(X−Nµ)2/2Nσ2 1 −1 = ∂X3+ ... 1 (N → ∞) . √N ξ, in which case N = ∂3 ∂X3 = N−1/2 ∂3 In going from the second line to the third, we have written X = gives a subleading contribution which vanishes in the N → ∞ limit. We have just proven the central limit theorem: in the limit N → ∞, the distribution of a sum of N independent random variables xiis a Gaussian with mean Nµ and standard deviation distribution P1(x). Note that P1(x) itself need not be a Gaussian – it could be a very peculiar distribution indeed, but so long as its first and second moment exist, where the kthmoment is simply hxki, the distribution of the sum X =PN ∂ξ3, which √N σ. Our only assumptions are that the mean µ and standard deviation σ exist for the i=1xiis a Gaussian. 1.4.3 Multidimensional Gaussian integral Consider the multivariable Gaussian distribution, ?detA ?1/2 ? ? −1 P(x) ≡ exp 2xiAijxj , (1.77) (2π)n where A is a positive definite matrix of rank n. A mathematical result which is extremely important throughout physics is the following: ?detA ?1/2 ∞ Z −∞ ∞ Z −∞ ? ? ? ? 1 2biA−1 −1 dx1··· Z(b) = dxnexp 2xiAijxj+ bixi = exp ijbj . (1.78) (2π)n Here, the vector b = (b1, ... , bn) is identified as a source. Since Z(0) = 1, we have that the distribution P(x) is normalized. Now consider averages of the form Z ∂nZ(b) ∂bj1··· ∂bj2k = A−1 dnx P(x) xj1··· xj2k hxj1···xj2ki = ????b=0 = (1.79) X jσ(1)jσ(2)···A−1 jσ(2k−1)jσ(2k). contractions The sum in the last term is over all contractions of the indices {j1, ... , j2k}. A contraction is an arrangement of the 2k indices into k pairs. There are C2k= (2k)!/2kk! possible such contractions. To obtain this result for Ck, we start with the first index and then find a mate among the remaining 2k − 1 indices. Then we choose the next unpaired index and find a mate among the remaining 2k − 3 indices. Proceeding in this manner, we have C2k= (2k − 1) · (2k − 3)···3 · 1 =(2k)! 2kk!. (1.80)

30. 1.5. APPENDIX : BAYESIAN STATISTICS 15 Equivalently, we can take all possible permutations of the 2k indices, and then divide by 2kk! since permuta- tion within a given pair results in the same contraction and permutation among the k pairs results in the same contraction. For example, for k = 2, we have C4= 3, and hxj1xj2xj3xj4i = A−1 j1j2A−1 j3j4+ A−1 j1j3A−1 j2j4+ A−1 j1j4A−1 j2j3. (1.81) 1.5 Appendix : Bayesian Statistics Let the probability of a discrete event A be P(A). We now introduce two additional probabilities. The joint probability for events A and B together is written P(A ∩ B). The conditional probability of B given A is P(B|A). We can compute the joint probability P(A ∩ B) = P(B ∩ A) in two ways: P(A ∩ B) = P(A|B) · P(B) = P(B|A) · P(A) . (1.82) Thus, P(A|B) =P(B|A) · P(A) , (1.83) P(B) a result known as Bayes’ theorem. Now suppose the ‘event space’ is partitioned as {Ai}. Then X P(B|Ai) · P(Ai) . P(B) = (1.84) i We then have P(B|Ai) · P(Ai) P P(Ai|B) = jP(B|Aj) · P(Aj), (1.85) a result sometimes known as the extended form of Bayes’ theorem. When the event space is a ‘binary partition’ {A,¬A}, we have P(A|B) = P(B|A) · P(A) P(B|A) · P(A) + P(B|¬A) · P(¬A). (1.86) Note that P(A|B) + P(¬A|B) = 1 (which follows from ¬¬A = A). As an example, consider the following problem in epidemiology. Suppose there is a rare but highly contagious disease A which occurs in 0.01% of the general population. Suppose further that there is a simple test for the disease which is accurate 99.99% of the time. That is, out of every 10,000 tests, the correct answer is returned 9,999 times, and the incorrect answer is returned only once8. Now let us administer the test to a large group of people from the general population. Those who test positive are quarantined. Question: what is the probability that someone chosen at random from the quarantine group actually has the disease? We use Bayes’ theorem with the binary partition {A,¬A}. Let B denote the event that an individual tests positive. Anyone from the quarantine group has tested positive. Given this datum, we want to know the probability that that person has the disease. That is, we want P(A|B). Applying eqn. 1.86 with P(¬A) = 0.9999 we find P(A|B) =1 despite the test being 99.99% accurate! The reason is that, given the rarity of the disease in the general population, the number of false positives is statistically equal to the number of true positives. P(B|A) = 0.9999 P(B|¬A) = 0.0001 , P(A) = 0.0001 , , , 2. That is, there is only a 50% chance that someone who tested positive actually has the disease, 8Epidemiologists define the sensitivity of a binary classification test as the fraction of actual positives which are correctly identified, and the specificity as the fraction of actual negatives that are correctly identified. In our example in the text, the sensitivity and specificity are both 0.9999.

31. 16 CHAPTER 1. PROBABILITY For continuous distributions, we speak of a probability density. We then have Z dx P(y|x) · P(x) P(y) = (1.87) and P(y|x) · P(x) Rdx′P(y|x′) · P(x′). P(x|y) = (1.88) The range of integration may depend on the specific application. The quantities P(Ai) are called the prior distribution. Clearly in order to compute P(B) or P(Ai|B) we must know the priors, and this is usually the weakest link in the Bayesian chain of reasoning. If our prior distribution is not accurate, Bayes’ theorem will generate incorrect results. One approach to obtaining the prior probabilities P(Ai) is to obtain them from a maximum entropy construction.

32. Chapter 2 Thermodynamics 2.1 References – E. Fermi, Thermodynamics (Dover, 1956) This outstanding and inexpensive little book is a model of clarity. – A. H. Carter, Classical and Statistical Thermodynamics (Benjamin Cummings, 2000) A very relaxed treatment appropriate for undergraduate physics majors. – H. B. Callen, Thermodynamics and an Introduction to Thermostatistics (2ndedition, Wiley, 1985) A comprehensive text appropriate for an extended course on thermodynamics. – D. V. Schroeder, An Introduction to Thermal Physics (Addison-Wesley, 2000) An excellent thermodynamics text appropriate for upper division undergraduates. Contains many illustra- tive practical applications. – D. Kondepudi and I. Prigogine, Modern Thermodynamics: From Heat Engines to Dissipative Structures (Wiley, 1998) Lively modern text with excellent choice of topics and good historical content. More focus on chemical and materials applications than in Callen. – L. E. Reichl, A Modern Course in Statistical Physics (2nd edition, Wiley, 1998) A graduate level text with an excellent and crisp section on thermodynamics. 17

33. 18 CHAPTER 2. THERMODYNAMICS 2.2 What is Thermodynamics? Thermodynamics is the study of relations among the state variables describing a thermodynamic system, and of transformations of heat into work and vice versa. 2.2.1 Thermodynamic systems and state variables Thermodynamic systems contain largenumbers of constituent particles, andaredescribedby a set of statevariables which describe the system’s properties in an average sense. State variables are classified as being either extensive or intensive. Extensive variables, such as volume V , particle number N, total internal energy E, magnetization M, etc., scale linearly with the system size, i.e. as the first power of the system volume. If we take two identical thermodynamic systems, place them next to each other, and remove any barriers between them, then all the extensive variables will double in size. Intensive variables, such as the pressurep, the temperature T, the chemical potential µ, the electric field E, etc., are independent of system size, scaling as the zeroth power of the volume. They are the same throughout the system, if that system is in an appropriate state of equilibrium. The ratio of any two extensive variables is an intensive variable. For example, we write n = N/V for the number density, which scales as V0. Intensive variables may also be inhomogeneous. For example, n(r) is the number density at position r, and is defined asthe limit of ∆N/∆V of the number of particles ∆N inside a volume ∆V which contains the point r, in the limit V ≫ ∆V ≫ V/N. Classically, the full motion of a system of N point particles requires 6N variables to fully describe it (3N positions and 3N velocities or momenta, in three space dimensions)1. Since the constituents are very small, N is typically very large. A typical solid or liquid, for example, has a mass density on the order of ̺ ∼ 1g/cm3; for gases, ̺ ∼ 10−3g/cm3. The constituent atoms have masses of 100to 102grams per mole, where one mole of X contains NAof X, and NA = 6.0221415 × 1023is Avogadro’s number. Thus, for solids and liquids we roughly expect number densities n of 10−2− 100mol/cm3for solids and liquids, and 10−5− 10−3mol/cm3for gases. Clearly we are dealing with fantastically large numbers of constituent particles in a typical thermodynamic system. The underlying theoretical basis for thermodynamics, where we use a small number of state variables to describe a system, is provided by the microscopic theory of statistical mechanics, which we shall study in the weeks ahead. Intensive quantities such as p, T, and n ultimately involve averages over both space and time. Consider for example the case of a gas enclosed in a container. We can measure the pressure (relative to atmospheric pressure) by attaching a spring to a moveable wall, as shown in Fig. 2.2. From the displacement of the spring and the value of its spring constant k we determine the force F. This force is due to the difference in pressures, so p = p0+F/A. Microscopically, the gas consists of constituent atoms or molecules, which are constantly undergoing collisions with each other and with the walls of the container. When a particle bounces off a wall, it imparts an impulse 2ˆ n(ˆ n · p), where p is the particle’s momentum and ˆ n is the unit vector normal to the wall. (Only particles with p · ˆ n > 0 will hit the wall.) Multiply this by the number of particles colliding with the wall per unit time, and one finds the net force on the wall; dividing by the area gives the pressure p. Within the gas, each particle travels for a distance ℓ, called the mean free path, before it undergoes a collision. We can write ℓ = ¯ vτ, where ¯ v is the average particle speed and τ is the mean free time. When we study the kinetic theory of gases, we will derive formulas for ℓ and ¯ v (and hence τ). For now it is helpful to quote some numbers to get an idea of the relevant distance and time scales. For O2gas at standard temperature and pressure (T = 0◦C, p = 1atm), the mean free path is ℓ ≈ 1.1×10−5cm, the average speed is ¯ v ≈ 480m/s, and the mean free time is τ ≈ 2.5×10−10s. Thus, particles in the gas undergo collisions at a rate τ−1≈ 4.0×109s−1. A measuring device, such as our spring, or a thermometer, 1For a system of N molecules which can freely rotate, we must then specify 3N additional orientational variables – the Euler angles – and their 3N conjugate momenta. The dimension of phase space is then 12N.

34. 2.2. WHAT IS THERMODYNAMICS? 19 Figure 2.1: From microscale to macroscale : physical versus social sciences. effectivelyperforms time and space averages. If there areNccollisions with a particularpatch of wall during some time interval on which our measurement device responds, then the root mean square relative fluctuations in the local pressure will be on the order of N−1/2 c times the average. Since Ncis a very large number, the fluctuations are negligible. If the system is in steady state, the state variables do not change with time. If furthermore there are no macroscopic currents of energy or particle number flowing through the system, the system is said to be in equilibrium. A continuous succession of equilibrium states is known as a thermodynamic path, which can be represented as a smooth curve in a multidimensional space whose axes are labeled by state variables. A thermodynamic process is any change or succession of changes which results in a change of the state variables. In a cyclic process, the initial and final states are the same. In a quasistatic process, the system passes through a continuous succession of equilibria. A reversible process is one where the external conditions and the thermodynamic path of the system can be reversed (at first this seems to be a tautology). All reversible processes are quasistatic, but not all quasistatic processes are reversible. For example, the slow expansion of a gas against a piston head, whose counter-force is always infinitesimally less than the force pA exerted by the gas, is reversible. To reverse this process, we simply add infinitesimally more force to pA and the gas compresses. A quasistatic process which is not reversible: slowly dragging a block across the floor, or the slow leak of air from a tire. Irreversibleprocesses, as arule, aredissipative. Other special processes include isothermal (dT = 0) isobaric (dp = 0), isochoric (dV = 0), and adiabatic (¯ dQ = 0, i.e. no heat exchange): reversible: ¯ dQ = T dS spontaneous: ¯ dQ < T dS adiabatic: ¯ dQ = 0 quasistatic: infinitely slowly isothermal: dT = 0 isochoric: dV = 0 isobaric: dp = 0 We shall discuss later the entropy S and its connection with irreversibility. How many state variables are necessary to fully specify the equilibrium state of a thermodynamic system? For a single component system, such as water which is composed of one constituent molecule, the answer is three. These can be taken to be T, p, and V . One always must specify at least one extensive variable, else we cannot determine the overall size of the system. For a multicomponent system with g different species, we must specify g + 2 state variables, which may be {T,p,N1,...,Ng}, where Nais the number of particles of species a. Another possibility is the set (T,p,V,x1,...,xg−1}, where the concentration of species a is xa= Na/N. Here, N =Pg If then follows that if we specify more than g + 2 state variables, there must exist a relation among them. Such relations are known as equations of state. The most famous example is the ideal gas law, a=1Na is the total number of particles. Note thatPg a=1xa= 1. pV = NkBT , (2.1)

35. 20 CHAPTER 2. THERMODYNAMICS Figure 2.2: The pressure p of a gas is due to an average over space and time of the impulses due to the constituent particles. relating the four state variables T, p, V , and N. Here kB= 1.3806503 × 10−16erg/K is Boltzmann’s constant. Another example is the van der Waals equation, ? where a and b are constants which depend on the molecule which forms the gas. For a third example, consider a paramagnet, where M V where M is the magnetization, H the magnetic field, and C the Curie constant. ? p +aN2 (V − bN) = NkBT , (2.2) V2 =CH , (2.3) T Any quantity which, in equilibrium, depends only on the state variables is called a state function. For example, the total internal energy E of a thermodynamics system is a state function, and we may write E = E(T,p,V ). State functions can also serve as state variables, although the most natural state variables are those which can be directly measured. 2.2.2 Heat Once thought to be a type of fluid, heat is now understood in terms of the kinetic theory of gases, liquids, and solids as a form of energy stored in the disordered motion of constituent particles. The units of heat are therefore units of energy, and it is appropriate to speak of heat energy, which we shall simply abbreviate as heat:2 1J = 107erg = 6.242× 1018eV = 2.390 × 10−4kcal = 9.478× 10−4BTU . We will use the symbol Q to denote the amount of heat energy absorbed by a system during some given ther- modynamic process, and ¯ dQ to denote a differential amount of heat energy. The symbol ¯ d indicates an ‘inexact differential’, about which we shall have more to say presently. This means that heat is not a state function: there is no ‘heat function’ Q(T,p,V ). (2.4) 2One calorie (cal) is the amount of heat needed to raise 1g of H2O from T0= 14.5◦C to T1= 15.5◦C at a pressure of p0= 1atm. One British Thermal Unit (BTU) is the amount of heat needed to raise 1lb. of H2O from T0= 63◦F to T1= 64◦F at a pressure of p0= 1atm.

36. 2.2. WHAT IS THERMODYNAMICS? 21 2.2.3 Work In general we will write the differential element of work ¯ dW done by the system as X ¯ dW = FidXi, (2.5) i where Fiis a generalized force and dXia generalized displacement3. The generalized forces and displacements are themselves state variables, and by convention we will take the generalized forces to be intensive and the general- ized displacements to be extensive. As an example, in a simple one-component system, we have ¯ dW = pdV . More generally, we write −P P jyjdXj }| aµadNa }| z { z { ?pdV − H · dM − E · dP − σ dA + ...?− ?µ1dN1+ µ2dN2+ ...? ¯ dW = (2.6) Here we distinguish between two types of work. The first involves changes in quantities such as volume, mag- netization, electric polarization, area, etc. The conjugate forces yiapplied to the system are then −p, the magnetic field H, the electric field E, the surface tension σ, respectively. The second type of work involves changes in the number of constituents of a given species. For example, energy is required in order to dissociate two hydrogen atoms in an H2molecule. The effect of such a process is dNH2= −1 and dNH= +2. As with heat, ¯ dW is an inexact differential, and work W is not a state variable, since it is path-dependent. There is no ‘work function’ W(T,p,V ). 2.2.4 Pressure and Temperature The units of pressure (p) are force per unit area. The SI unit is the Pascal (Pa): 1Pa = 1N/m2= 1kg/ms2. Other units of pressure we will encounter: 1bar ≡ 105Pa 1atm ≡ 1.01325× 105Pa 1torr ≡ 133.3Pa . Temperature(T)has a very precisedefinition from the point of view of statistical mechanics, as we shall see. Many physical properties depend on the temperature – such properties are called thermometric properties. For example, the resistivity of a metal ρ(T,p) or the number density of a gas n(T,p) are both thermometric properties, and can be used to define a temperature scale. Consider the device known as the ‘constant volume gas thermometer’ depicted in Fig. 2.3, in which the volume or pressure of a gas may be used to measure temperature. The gas is assumed to be in equilibrium at some pressure p, volume V , and temperature T. An incompressible fluid of density ̺ is used to measure the pressure difference ∆p = p − p0, where p0is the ambient pressure at the top of the reservoir: p − p0= ̺g(h2− h1) , where g is the acceleration due to gravity. The height h1of the left column of fluid in the U-tube provides a measure of the change in the volume of the gas: (2.7) V (h1) = V (0) − Ah1, (2.8) where A is the (assumed constant) cross-sectional area of the left arm of the U-tube. The device can operate in two modes: 3We use the symbol ¯ d in the differential ¯ dW to indicate that this is not an exact differential. More on this in section 2.4 below.

37. 22 CHAPTER 2. THERMODYNAMICS Figure 2.3: The constant volume gas thermometer. The gas is placed in thermal contact with an object of temper- ature T. An incompressible fluid of density ̺ is used to measure the pressure difference ∆p = pgas− p0. • Constant pressure mode : The height of the reservoir is adjusted so that the height difference h2−h1is held constant. This fixes the pressure p of the gas. The gas volume still varies with temperature T, and we can define T Tref V = , (2.9) Vref where Trefand Vrefare the reference temperature and volume, respectively. • Constant volume mode : The height of the reservoir is adjusted so that h1= 0, hence the volume of the gas is held fixed, and the pressure varies with temperature. We then define T p = , (2.10) Tref pref where Trefand prefare the reference temperature and pressure, respectively. What should we use for a reference? One might think that a pot of boiling water will do, but anyone who has gone camping in the mountains knows that water boils at lower temperatures at high altitude (lower pressure). This phenomenon is reflected in the phase diagram for H2O, depicted in Fig. 2.4. There are two special points in the phase diagram, however. One is the triple point, where the solid, liquid, and vapor (gas) phases all coexist. The second is the critical point, which is the terminus of the curve separating liquid from gas. At the critical point, the latent heatof transition between liquid andgas phases vanishes (more on this later on). The triple point temperature Ttat thus unique and is by definition Tt= 273.16K. The pressure at the triple point is 611.7Pa = 6.056 × 10−3atm. A question remains: are the two modes of the thermometer compatible? E.g. it we boil water at p = p0= 1atm, do they yield the same value for T? And what if we use a different gas in our measurements? In fact, all these measurements will in general be incompatible, yielding different results for the temperature T. However, in the limit that we use a very low density gas, all the results converge. This is because all low density gases behave as ideal gases, and obey the ideal gas equation of state pV = NkBT.

38. 2.2. WHAT IS THERMODYNAMICS? 23 Figure 2.4: A sketch of the phase diagram of H2O (water). Two special points areidentified: the triple point (Tt,pt) at which there is three phase coexistence, and the critical point (Tc,pc), where the latent heat of transformation from liquid to gas vanishes. Not shown are transitions between several different solid phases. 2.2.5 Standard temperature and pressure It is customary in the physical sciences to define certain standard conditions with respect to which any arbitrary conditions may be compared. In thermodynamics, there is a notion of standard temperature and pressure, abbre- viated STP. Unfortunately, there are two different definitions of STP currently in use, one from the International Union of Pure and Applied Chemistry (IUPAC), and the other from the U.S. National Institute of Standards and Technology (NIST). The two standards are: IUPAC : T0= 0◦C = 273.15K NIST : T0= 20◦C = 293.15K p0= 105Pa p0= 1atm = 1.01325× 105Pa , , To make matters worse, in the past it was customary to define STP as T0= 0◦C and p0= 1atm. We will use the NIST definition in this course. Unless I slip and use the IUPAC definition. Figuring out what I mean by STP will keep you on your toes. The volume of one mole of ideal gas at STP is then ( V =NAkBT0 22.711ℓ 24.219ℓ (IUPAC) (NIST) , = (2.11) p0 where 1ℓ = 106cm3= 10−3m3is one liter. Under the old definition of STP as T0= 0◦C and p0= 1atm, the volume of one mole of gas at STP is 22.414 ℓ, which is a figure I remember from my 10thgrade chemistry class with Mr. Lawrence.

39. 24 CHAPTER 2. THERMODYNAMICS Figure 2.5: As the gas density tends to zero, the readings of the constant volume gas thermometer converge. 2.3 The Zeroth Law of Thermodynamics Equilibrium is established by the exchange of energy, volume, or particle number between different systems or subsystems: =⇒ =⇒ T = constant energy exchange thermal equilibrium =⇒ =⇒ p = constant volume exchange mechanical equilibrium =⇒ =⇒ µ = constant particle exchange chemical equilibrium Equilibrium is transitive, so If A is in equilibrium with B, and B is in equilibrium with C, then A is in equilibrium with C. This known as the Zeroth Law of Thermodynamics4. 2.4 Mathematical Interlude : Exact and Inexact Differentials The differential k X dF = Aidxi (2.12) i=1 is called exact if there is a function F(x1,...,xk) whose differential gives the right hand side of eqn. 2.12. In this case, we have Ai=∂F ∂xi ∂Ai ∂xj =∂Aj ∂xi ⇐⇒ ∀ i,j . (2.13) 4As we shall see further below, thermomechanical equilibrium in fact leads to constant p/T, and thermochemical equilibrium to constant µ/T. If there is thermal equilibrium, then T is already constant, and so thermomechanical and thermochemical equilibria then guarantee the constancy of p and µ.

40. 2.4. MATHEMATICAL INTERLUDE : EXACT AND INEXACT DIFFERENTIALS 25 Figure 2.6: Two distinct paths with identical endpoints. For exact differentials, the integral between fixed endpoints is path-independent: Z A B dF = F(xB 1,...,xB k) − F(xA 1,...,xA k) , (2.14) from which it follows that the integral of dF around any closed path must vanish: I dF = 0 . (2.15) When the cross derivatives are not identical, i.e. when ∂Ai/∂xj6= ∂Aj/∂xi, the differential is inexact. In this case, the integral of dF is path dependent, and does not depend solely on the endpoints. As an example, consider the differential dF = K1ydx + K2xdy . (2.16) Let’s evaluate the integral of dF, which is the work done, along each of the two paths in Fig. 2.6: xB Z xA yB Z yA W(I)= K1 dy xB= K1yA(xB− xA) + K2xB(yB− yA) dx yA+ K2 (2.17) xB Z xA yB Z yA W(II)= K1 dy xA= K1yB(xB− xA) + K2xA(yB− yA) . dx yB+ K2 (2.18) Note that in general W(I)6= W(II). Thus, if we start at point A, the kinetic energy at point B will depend on the path taken, since the work done is path-dependent.

41. 26 CHAPTER 2. THERMODYNAMICS Figure 2.7: The first law of thermodynamics is a statement of energy conservation. The difference between the work done along the two paths is I W(I)− W(II)= dF = (K2− K1)(xB− xA)(yB− yA) . (2.19) Thus, we see that if K1= K2, the work is the same for the two paths. In fact, if K1= K2, the work would be path-independent, and would depend only on the endpoints. This is true for any path, and not just piecewise linear paths of the type depicted in Fig. 2.6. Thus, if K1= K2, we are justified in using the notation dF for the differential in eqn. 2.16; explicitly, we then have F = K1xy. However, if K16= K2, the differential is inexact, and we will henceforth write ¯ dF in such cases. 2.5 The First Law of Thermodynamics 2.5.1 Conservation of energy The first law is a statement of energy conservation, and is depicted in Fig. 2.7. It says, quite simply, that during a thermodynamic process, the change in a system’s internal energy E is given by the heat energy Q added to the system, minus the work W done by the system: ∆E = Q − W . (2.20) The differential form of this, the First Law of Thermodynamics, is dE = ¯ dQ − ¯ dW . (2.21) We use the symbol ¯ d in the differentials ¯ dQ and ¯ dW to remind us that these are inexact differentials. The energy E, however, is a state function, hence dE is an exact differential. Consider a volume V of fluid held in a flask, initially at temperature T0, and held at atmospheric pressure. The internal energy is then E0= E(T0,p,V ). Now let us contemplate changing the temperature in two different ways. The first method (A) is to place the flask on a hot plate until the temperature of the fluid rises to a value T1. The second method (B) is to stir the fluid vigorously. In the first case, we add heat QA> 0 but no work is done, so WA= 0. In the second case, if we thermally insulate the flask and use a stirrer of very low thermal conductivity, then no heat is added, i.e. QB= 0. However, the stirrer does work −WB> 0 on the fluid (remember W is the work done by the system). If we end up at the same temperature T1, then the final energy is E1= E(T1,p,V ) in both cases. We then have ∆E = E1− E0= QA= −WB. (2.22) It also follows that for any cyclic transformation, where the state variables are the same at the beginning and the end, we have ∆Ecyclic= Q − W = 0 =⇒ Q = W (cyclic) . (2.23)

42. 2.5. THE FIRST LAW OF THERMODYNAMICS 27 2.5.2 Single component systems Asingle component system is specified bythreestatevariables. In many applications, the totalnumber of particles N is conserved, so it is useful to take N as one of the state variables. The remaining two can be (T,V ) or (T,p) or (p,V ). The differential form of the first law says dE = ¯ dQ − ¯ dW = ¯ dQ − pdV + µdN . (2.24) The quantity µ is called the chemical potential. Here we shall be interested in the case dN = 0 so the last term will not enter into our considerations. We ask: how much heat is required in order to make an infinitesimal change in temperature, pressure, or volume? We start by rewriting eqn. 2.24 as ¯ dQ = dE + pdV − µdN . (2.25) We now must roll up our sleeves and do some work with partial derivatives. • (T,V,N) systems : If the state variables are (T,V,N), we write ?∂E ? ?∂E ? ?∂E ? dE = dT + dV + dN . (2.26) ∂T ∂V ∂N V,N T,N T,V Then "?∂E # "?∂E # ?∂E ? ? ? − µ ¯ dQ = dT + + p dV + dN . (2.27) ∂T ∂V ∂N V,N T,N T,V • (T,p,N) systems : If the state variables are (T,p,N), we write ?∂E ? ?∂E ? ?∂E ? dE = dT + dp + dN . (2.28) ∂T ∂p ∂N p,N T,N T,p We also write ?∂V ? ?∂V ? ?∂V ? dV = dT + dp + dN . (2.29) ∂T ∂p ∂N p,N T,N T,p Then "?∂E # "?∂E # ? ?∂V ? ? ? ?∂V ? ¯ dQ = + p dT + + p dp ∂T ∂T ∂p ∂p p,N "?∂E p,N T,N T,N # (2.30) ?∂V ? − µ + + p dN . ∂N ∂N T,p T,p • (p,V,N) systems : If the state variables are (p,V,N), we write ?∂E ? ?∂E ? ?∂E ? dE = dp + dV + dN . (2.31) ∂p ∂V ∂N V,N p,N p,V Then "?∂E # "?∂E # ?∂E ? ? ? − µ ¯ dQ = dp + + p dV + dN . (2.32) ∂p ∂V ∂N V,N p,N p,V

43. 28 CHAPTER 2. THERMODYNAMICS cp ˜ cp cp ˜ cp (J/molK) 29.07 24.2 24.47 36.94 6.115 112 25.42 20.786 28.82 38.09 (J/gK) 1.01 0.897 0.385 0.839 0.509 2.44 0.129 5.193 5.19 2.05 (J/molK) 75.34 37.47 25.1 26.4 24.8 20.786 29.38 900 27.7 25.3 (J/gK) 4.181 2.08 0.450 0.127 3.58 1.03 0.918 2.5 0.116 0.387 SUBSTANCE Air Aluminum Copper CO2 Diamond Ethanol Gold Helium Hydrogen H2O (−10◦C) SUBSTANCE H2O (25◦C) H2O (100◦ Iron Lead Lithium Neon Oxygen Paraffin (wax) Uranium Zinc +C) Table 2.1: Specific heat (at 25◦C, unless otherwise noted) of some common substances. (Source: Wikipedia.) The heat capacity of a body, C, is by definition the ratio ¯ dQ/dT of the amount of heat absorbed by the body to the associated infinitesimal change in temperature dT. The heat capacity will in general be different if the body is heated at constant volume or at constant pressure. Setting dV = 0 gives, from eqn. 2.27, ?¯ dQ ? ?∂E ? CV,N= = . (2.33) dT ∂T V,N V,N Similarly, if we set dp = 0, then eqn. 2.30 yields ?¯ dQ ? ?∂E ? ?∂V ? Cp,N= = + p . (2.34) dT ∂T ∂T p,N p,N p,N Unless explicitly stated as otherwise, we shall assume that N is fixed, and will write CVfor CV,Nand Cpfor Cp,N. The units of heat capacity are energy divided by temperature, e.g. J/K. The heat capacity is an extensive quantity, scaling with the size of the system. If we divide by the number of moles N/NA, we obtain the molar heat capacity, sometimes called the molar specific heat: c = C/ν, where ν = N/NAis the number of moles of substance. Specific heat is also sometimes quoted in units of heat capacity per gram of substance. We shall define C c M=heat capacity per mole mass per mole ˜ c = mN= . (2.35) Here m is the mass per particle and M is the mass per mole: M = NAm. Suppose we raise the temperature of a body from T = TAto T = TB. How much heat is required? We have TB Z TA Q = dT C(T) , (2.36) where C = CVor C = Cpdepending on whether volume or pressure is held constant. For ideal gases, as we shall discuss below, C(T) is constant, and thus TB= TA+Q Q = C(TB− TA) =⇒ C. (2.37)

44. 2.5. THE FIRST LAW OF THERMODYNAMICS 29 Figure 2.8: Heat capacity CV for one mole of hydrogen (H2) gas. At the lowest temperatures, only translational degreesof freedom arerelevant, and f = 3. At around 200K,two rotational modes areexcitable and f = 5. Above 1000K, the vibrational excitations begin to contribute. Note the logarithmic temperature scale. (Data from H. W. Wooley et al., Jour. Natl. Bureau of Standards, 41, 379 (1948).) In metals at very low temperatures one finds C = γT, where γ is a constant5. We then have TB Z TA p 2γ?T2 ? B− T2 dT C(T) =1 Q = (2.38) A T2 A+ 2γ−1Q . TB= (2.39) 2.5.3 Ideal gases The ideal gas equation of state is pV = NkBT. In order to invoke the formulae in eqns. 2.27, 2.30, and 2.32, we need to know the state function E(T,V,N). A landmark experiment by Joule in the mid-19th century established that the energy of a low density gas is independent of its volume6. Essentially, a gas at temperatureT was allowed to freely expand from one volume V to a larger volume V′> V , with no added heat Q and no work W done. Therefore the energy cannot change. What Joule found was that the temperature also did not change. This means that E(T,V,N) = E(T,N) cannot be a function of the volume. Since E is extensive, we conclude that E(T,V,N) = ν ε(T) , (2.40) where ν = N/NAis the number of moles of substance. Note that ν is an extensive variable. From eqns. 2.33 and 2.34, we conclude CV(T) = ν ε′(T) , Cp(T) = CV(T) + νR , (2.41) where we invoke the ideal gas law to obtain the second of these. Empirically it is found that CV(T) is temperature independentoverawiderangeofT,farenoughfromboiling point. Wecanthen writeCV= ν cV, whereν ≡ N/NA is the number of moles, and where cVis the molar heat capacity. We then have cp= cV+ R , (2.42) 5In most metals, the difference between CVand Cpis negligible. 6See the description in E. Fermi, Thermodynamics, pp. 22-23.

45. 30 CHAPTER 2. THERMODYNAMICS Figure 2.9: Molar heat capacities cV for three solids. The solid curves correspond to the predictions of the Debye model, which we shall discuss later. where R = NAkB= 8.31457J/molK is the gas constant. We denote by γ = cp/cVthe ratio of specific heat at constant pressure and at constant volume. From the kinetic theory of gases, one can show that cV=3 cV=5 cV= 3R cp=5 cp=7 cp= 4R γ =5 γ =7 γ =4 2R , 2R , monatomic gases: 3 2R , 2R , diatomic gases: 5 , , 3. polyatomic gases: Digression : kinetic theory of gases We will conclude in general from noninteracting classical statistical mechanics that the specific heat of a substance is cv=1 or potential energy function. For example, a point particle has three translational degrees of freedom, and the kinetic energy is a quadratic function of their conjugate momenta: H0= (p2 molecules have two additional rotational degrees of freedom – we don’t count rotations about the symmetry axis – and their conjugate momenta also appear quadratically in the kinetic energy, leading to f = 5. For polyatomic molecules, all three Euler angles and their conjugate momenta are in play, and f = 6. 2fR, where f is the number of phase space coordinates, per particle, for which there is a quadratic kinetic x+p2 y+p2 z)/2m. Thus, f = 3. Diatomic The reason that f = 5 for diatomic molecules rather than f = 6 is due to quantum mechanics. While translational eigenstates form a continuum, or arequantized in abox with ∆kα= 2π/Lαbeing very small, since the dimensions Lαare macroscopic, angular momentum, and hence rotational kinetic energy, is quantized. For rotations about a principal axis with very low moment of inertia I, the corresponding energy scale ~2/2I is very large, and a high temperature is required in order to thermally populate these states. Thus, degrees of freedom with a quantization energy on the order or greater than ε0are ‘frozen out’ for temperatures T<∼ε0/kB. In solids, each atom is effectively connected to its neighbors by springs; such a potential arises from quantum mechanical and electrostatic consideration of the interacting atoms. Thus, each degree of freedom contributes to the potential energy, and its conjugate momentum contributes to the kinetic energy. This results in f = 6. Assuming only lattice vibrations, then, the high temperature limit for cV(T) for any solid is predicted to be 3R = 24.944J/molK. This is calledthe Dulong-Petitlaw. The high temperaturelimit is reachedabove the so-called Debye

46. 2.5. THE FIRST LAW OF THERMODYNAMICS 31 temperature, which is roughly proportional to the melting temperature of the solid. In table 2.1, we list cpand ˜ cpfor some common substances at T = 25◦C (unless otherwise noted). Note that cpfor the monatomic gases He and Ne is to high accuracy given by the value from kinetic theory, cp= 20.7864J/molK. For the diatomic gases oxygen (O2) and air (mostly N2and O2), kinetic theory predicts cp= 7 2R = 29.10, which is close to the measuredvalues. Kinetic theory predicts cp= 4R = 33.258for polyatomic gases; the measured values for CO2and H2O are both about 10% higher. 5 2R = 2.5.4 Adiabatic transformations of ideal gases Assuming dN = 0 and E = ν ε(T), eqn. 2.27 tells us that ¯ dQ = CVdT + pdV . (2.43) Invoking the ideal gas law to write p = νRT/V , and remembering CV= ν cV, we have, setting ¯ dQ = 0, dT T +R dV V = 0 . (2.44) cV We can immediately integrate to obtain   TVγ−1= constant pVγ= constant Tγp1−γ= constant    =⇒ ¯ dQ = 0 (2.45) where the second two equations are obtained from the first by invoking the ideal gas law. These are all adiabatic equations of state. Note the difference between the adiabatic equation of state d(pVγ) = 0 and the isothermal equation of state d(pV ) = 0. Equivalently, we can write these three conditions as 0Tf pfVf+2= pf 0Vf+2 0 Tf+2p−2= Tf+2 p−2 0 V2Tf= V2 , , . (2.46) 0 0 It turns out thatairis aratherpoor conductor ofheat. This suggests thefollowing model for anadiabaticatmosphere. The hydrostatic pressure decrease associated with an increase dz in height is dp = −̺gdz, where ̺ is the density and g the acceleration due to gravity. Assuming the gas is ideal, the density can be written as ̺ = Mp/RT, where M is the molar mass. Thus, dp p If the height changes are adiabatic, then, from d(Tγp1−γ) = 0, we have = −Mg RTdz . (2.47) dT =γ − 1 = −γ − 1 Tdp p Mg R dz , (2.48) γ γ with the solution ? ? T(z) = T0−γ − 1 1 −γ − 1 Mg R z λ z = T0, (2.49) γ γ where T0= T(0) is the temperature at the earth’s surface, and λ =RT0 Mg. (2.50) With M = 28.88gandγ =7 −9.7K/km. Note that in this model the atmosphere ends at a height zmax= γλ/(γ − 1) = 30km. 5for air,andassuming T0= 293K,wefind λ = 8.6km, anddT/dz = −(1−γ−1)T0/λ =

47. 32 CHAPTER 2. THERMODYNAMICS Again invoking the adiabatic equation of state, we can find p(z): ?T ? ? ? γ γ 1 −γ − 1 p(z) p0 z λ γ−1 γ−1 = = (2.51) T0 γ Recall that ? ?k 1 +x ex= lim . (2.52) k k→∞ Thus, in the limit γ → 1, where k = γ/(γ −1) → ∞, we have p(z) = p0exp(−z/λ). Finally, since ̺ ∝ p/T from the ideal gas law, we have ? ? 1 1 −γ − 1 ̺(z) ̺0 z λ γ−1 = . (2.53) γ 2.5.5 Adiabatic free expansion Consider the situation depicted in Fig. 2.10. A quantity (ν moles) of gas in equilibrium at temperature T and volume V1is allowed to expand freely into an evacuated chamber of volume V2by the removal of a barrier. Clearly no work is done on or by the gas during this process, hence W = 0. If the walls are everywhere insulating, so that no heat can pass through them, then Q = 0 as well. The First Law then gives ∆E = Q − W = 0, and there is no change in energy. If the gas is ideal, then since E(T,V,N) = NcVT, then ∆E = 0 gives ∆T = 0, and there is no change in tem- perature. (If the walls are insulating against the passage of heat, they must also prevent the passage of particles, so ∆N = 0.) There is of course a change in volume: ∆V = V2, hence there is a change in pressure. The initial pressure is p = NkBT/V1and the final pressure is p′= NkBT/(V1+ V2). If the gas is nonideal, then the temperature will in general change. Suppose, for example, that E(T,V,N) = αVxN1−xTy, whereα, x, andy areconstants. Thisform is properlyextensive: if V andN double, then E doubles. If the volume changes from V to V′under an adiabatic free expansion, then we must have, from ∆E = 0, ?V ?x ?T′ ?y ?V ?x/y T′= T · =⇒ = . (2.54) V′ V′ T If x/y > 0, the temperature decreases upon the expansion. If x/y < 0, the temperature increases. Without an equation of state, we can’t say what happens to the pressure. Adiabaticfreeexpansionofagasisaspontaneousprocess, arisingduetothenaturalinternaldynamicsofthesystem. It is also irreversible. If we wish to take the gas back to its original state, we must do work on it to compress it. If the gas is ideal, then the initial and final temperatures are identical, so we can place the system in thermal contact with a reservoir at temperature T and follow a thermodynamic path along an isotherm. The work done on the gas during compression is then Vi ?Vf ? ? ? Z Vf dV V 1 +V2 W = −NkBT = NkBT ln = NkBT ln (2.55) Vi V1 The work done by the gas is W =RpdV = −W. During the compression, heat energy Q = W < 0 is transferred to the gas from the reservoir. Thus, Q = W > 0 is given off by the gas to its environment.

48. 2.6. HEAT ENGINES AND THE SECOND LAW OF THERMODYNAMICS 33 Figure 2.10: In the adiabatic free expansion of a gas, there is volume expansion with no work or heat exchange with the environment: ∆E = Q = W = 0. 2.6 Heat Engines and the Second Law of Thermodynamics 2.6.1 There’s no free lunch so quit asking A heat engine is a device which takes a thermodynamic system through a repeated cycle which can be represented as a succession of equilibrium states: A → B → C··· → A. The net result of such a cyclic process is to convert heat into mechanical work, or vice versa. For a system in equilibrium at temperature T, there is a thermodynamically large amount of internal energy stored in the random internal motion of its constituent particles. Later, when we study statistical mechanics, we will see how each ‘quadratic’ degree of freedom in the Hamiltonian contributes1 An immense body in equilibrium at temperature T has an enormous heat capacity C, hence extracting a finite quantity of heat Q from it results in a temperature change ∆T = −Q/C which is utterly negligible. Such a body is called a heat bath, or thermal reservoir. A perfect engine would, in each cycle, extract an amount of heat Q from the bath and convert it into work. Since ∆E = 0 for a cyclic process, the First Law then gives W = Q. This situation is depicted schematically in Fig. 2.11. One could imagine running this process virtually indefinitely, slowly sucking energy out of an immense heat bath, converting the random thermal motion of its constituent molecules into useful mechanical work. Sadly, this is not possible: 2kBT to the total internal energy. A transformation whose only final result is to extract heat from a source at fixed temperature and transform that heat into work is impossible. This is known as the Postulate of Lord Kelvin. It is equivalent to the postulate of Clausius, A transformation whose only result is to transfer heat from a body at a given temperature to a body at higher temperature is impossible. These postulates which have been repeatedly validated by empirical observations, constitute the Second Law of Thermodynamics.

49. 34 CHAPTER 2. THERMODYNAMICS Figure 2.11: A perfect engine would extract heat Q from a thermal reservoir at some temperature T and convert it into useful mechanical work W. This process is alas impossible, according to the Second Law of thermodynamics. The inverse process, where work W is converted into heat Q, is always possible. 2.6.2 Engines and refrigerators While it is not possible to convert heat into work with 100% efficiency, it is possible to transfer heat from one thermal reservoir to another one, at lower temperature, and to convert some of that heat into work. This is what an engine does. The energy accounting for one cycle of the engine is depicted in the left hand panel of Fig. 2.12. An amount of heat Q2> 0 is extracted- from the reservoir at temperature T2. Since the reservoir is assumed to be enormous, its temperature change ∆T2= −Q2/C2is negligible, and its temperature remains constant – this is what it means for an object to be a reservoir. A lesser amount of heat, Q1, with 0 < Q1< Q2, is deposited in a second reservoir at a lower temperature T1. Its temperature change ∆T1= +Q1/C1is also negligible. The difference W = Q2− Q1is extracted as useful work. We define the efficiency, η, of the engine as the ratio of the work done to the heat extracted from the upper reservoir, per cycle: = 1 −Q1 η =W . (2.56) Q2 Q2 This is a natural definition of efficiency, since it will cost us fuel to maintain the temperature of the upper reservoir over many cycles of the engine. Thus, the efficiency is proportional to the ratio of the work done to the cost of the fuel. A refrigerator works according to the same principles, but the process runs in reverse. An amount of heat Q1is extracted from the lower reservoir – the inside of our refrigerator – and is pumped into the upper reservoir. As Clausius’ form of the Second Law asserts, it is impossible for this to be the only result of our cycle. Some amount of work W must be performed on the refrigerator in order for it to extract the heat Q1. Since ∆E = 0 for the cycle, a heat Q2= W + Q1must be deposited into the upper reservoir during each cycle. The analog of efficiency here is called the coefficient of refrigeration, κ, defined as κ =Q1 Q1 W= . (2.57) Q2− Q1 Thus, κ is proportional to the ratio of the heat extracted to the cost of electricity, per cycle. Please note the deliberate notation here. I am using symbols Q and W to denote the heat supplied to the engine (or refrigerator) and the work done by the engine, respectively, and Q and W to denote the heat taken from the engine and the work done on the engine. A perfect engine has Q1= 0 and η = 1; a perfect refrigerator has Q1= Q2and κ = ∞. Both violate the Second Law. Sadi Carnot7(1796 – 1832) realized that a reversible cyclic engine operating between two thermal reservoirs 7Carnot died during cholera epidemic of 1832. His is one of the 72 names engraved on the Eiffel Tower.

50. 2.6. HEAT ENGINES AND THE SECOND LAW OF THERMODYNAMICS 35 Figure 2.12: An engine (left) extracts heat Q2from a reservoir at temperature T2and deposits a smaller amount of heat Q1into a reservoir at a lower temperature T1, during each cycle. The difference W = Q2−Q1is transformed into mechanical work. A refrigerator (right) performs the inverse process, drawing heat Q1from a low tempera- ture reservoir and depositing heat Q2= Q1+W into a high temperature reservoir, where W is the mechanical (or electrical) work done per cycle. must produce the maximum amount of work W, and that the amount of work produced is independent of the material properties of the engine. We call any such engine a Carnot engine. The efficiency of a Carnot engine may be used to define a temperature scale. We know from Carnot’s observations that the efficiency ηCcan only be a function of the temperatures T1and T2: ηC= ηC(T1,T2). We can then define T1 T2 ≡ 1 − ηC(T1,T2) . (2.58) Below, in §2.6.4, we will see that how, using an ideal gas as the ‘working substance’ of the Carnot engine, this temperature scale coincides precisely with the ideal gas temperature scale from §2.2.4. 2.6.3 Nothing beats a Carnot engine The Carnot engine is the most efficient engine possible operating between two thermal reservoirs. To see this, let’s suppose that an amazing wonder engine has an efficiency even greater than that of the Carnot engine. A key feature of the Carnot engine is its reversibility – we can just go around its cycle in the opposite direction, creating a Carnot refrigerator. Let’s use our notional wonder engine to drive a Carnot refrigerator, as depicted in Fig. 2.13. We assume that = ηwonder> ηCarnot=W′ W Q2 . (2.59) Q′ 2 But from the figure, we have W = W′, and therefore the heat energy Q′ positive. From 2−Q2transferred to the upper reservoir is W = Q2− Q1= Q′ 2− Q′ 1= W′, (2.60) we see that this is equal to the heat energy extracted from the lower reservoir, since no external work is done on the system: Q′ 2− Q2= Q′ 1− Q1> 0 . (2.61)