Derivations of the Boltzmann Distribution

Derivation by the Limit of the Bath's Heat Capacity In the Canonical Ensemble

In the Canonical ensemble the combined energy of the bath and system should be constant. The combination of the 2 makes a microcanonical ensemble. Let $W_{total} (E_{total})$ be the number of accessable states when the total energy is equal to $E_{total}$ , and the same with $W_{bath}$ and $W_{sys}$ . Let the set of microstates of the system can be in $X_{sys}$ , the microstates the bath can be in $X_{bath}$ , and the total microstates $X_{total} = X_{bath} \times X_{sys}$ . The total energy should be the sum of the two energies plus a coupling energy.

E_{total} = E_{bath} + E_{sys} + E_{coupling}

The total accessable states with an energy should be the total number combined number of states the bath and system can be in that sum to that energy.

W_{total} (E_{total}) = \sum_{r \in X} W_{bath} (E_{total} - E_{sys} (r))

And in the microcanonical ensemble all microstates $(r_{bath}, r_{sys}) \in X_{total}$ with the correct have an equal probaility.

p (r_{bath}, r_{sys}) = \frac{1}{W_{total} (E_{total})} 𝟙 (E_{total} = E_{sys} (r_{sys}) + E_{bath} (r_{bath}))

We can calculate the probability of just the microstate for the system.

p (r_{sys}) = \sum_{r_{bath} \in X_{bath}} p (r_{bath}, r_{sys})

= \sum_{r_{bath} \in X_{bath}} \frac{1}{W_{total} (E_{total})} 𝟙 (E_{total} = E_{sys} (r_{sys}) + E_{bath} (r_{bath}))

= \frac{W_{bath} (E_{total} - E_{sys} (r_{sys}))}{W_{total} (E_{total})}

Take the natural log of both sides

\ln (p (r_{sys})) = \ln (\frac{W_{bath} (E_{total} - E_{sys} (r_{sys}))}{W_{total} (E_{total})})

\ln (p (r_{sys})) = \ln (W_{bath} (E_{total} - E_{sys} (r_{sys}))) - \ln (W_{total} (E_{total}))

Then we expand the logarithm to its Taylor series.

f (x + c) = f (x) + \frac{d f (x)}{d x} c + \frac{1}{2} \frac{d^{2} f (x)}{d x^{2}} c^{2} + \frac{1}{6} \frac{d^{3} f (x)}{d x^{3}} c^{3} + \dots

We set $f (x) = \ln (W_{bath} (x))$ , $x = E_{total}$ , and $c = - E_{sys}$ we get the following expression.

\ln (p (r_{sys})) = \ln (W_{bath} (E_{total})) - \frac{d \ln (W_{bath} (E_{total}))}{d E_{total}} E_{sys} (r_{sys})

+ \frac{1}{2} \frac{d^{2} \ln (W_{bath} (E_{total}))}{d E_{total}^{2}} E_{sys} (r_{sys})^{2} - \frac{1}{6} \frac{d^{3} \ln (W_{bath} (E_{total}))}{d E_{total}^{3}} E_{sys} (r_{sys})^{3} + \dots

\dots - \ln (W_{total} (E_{total}))

In the limit that $E_{total}$ approaches $E_{bath}$ the bath becomes a microcanonical ensemble and the entropy of the bath $S_{bath} (E_{total}) = S_{bath} (E_{bath}) = k_{B} \ln (W_{bath} (E_{bath}))$

= \ln (W_{bath} (E_{bath})) - \frac{1}{k_{B}} \frac{d S_{bath} (E_{bath})}{d E_{bath}} E_{sys} (r_{sys}) + \frac{1}{2 k_{B}} \frac{d^{2} S_{bath} (E_{bath})}{d E_{bath}^{2}} E_{sys} (r_{sys})^{2}

- \frac{1}{6 k_{B}} \frac{d^{3} S_{bath} (E_{bath})}{d E_{bath}^{3}} E_{sys} (r_{sys})^{3} + \dots - \ln (W_{total} (E_{total}))

And by definition $\frac{1}{T_{bath}} = \frac{d S_{bath} (E)}{d E}$ .

= \ln (W_{bath} (E_{bath})) - \frac{1}{k_{B} T_{bath}} E_{sys} (r_{sys}) + \frac{1}{2 k_{B} T_{bath}} \frac{d T_{bath}}{d E_{bath}^{2}} E_{sys} (r_{sys})^{2}

- \frac{1}{6 k_{B} T_{bath}} \frac{d^{2} T_{bath}}{d E_{bath}^{2}} E_{sys} (r_{sys})^{3} + \dots - \ln (W_{total} (E_{total}))

Then if we assume the heat capacity of the bath is infinite so that $\frac{d T_{bath}}{d E_{bath}} = 0$ we can remove the extra terms.

\ln (p (r_{sys})) = - \frac{1}{k_{B} T_{bath}} E_{sys} (r_{sys}) + \ln (\frac{W_{bath} (E_{bath})}{W_{total} (E_{total})})

Then exponentiate both sides.

p (r_{sys}) = \frac{e^{- \frac{E_{sys} (r_{sys})}{k_{B} T_{bath}}}}{(\frac{W_{total} (E_{total})}{W_{bath} (E_{bath})})}

And the denominator is just a normalizing factor so the final expression is below.

p (r_{sys}) = \frac{e^{- \frac{E_{sys} (r_{sys})}{k_{B} T_{bath}}}}{\sum_{r_{sys}^{'} \in X_{sys}} e^{- \frac{E_{sys} (r_{sys}^{'})}{k_{B} T_{bath}}}}

Derivation of the Boltzmann Distribution by Maximizing Entropy

Assume there are $N$ discrete states the system can be in: $X = {x_{1} \dots x_{N}}$ . Let $p_{i} = P (X = x_{i})$ sampling the state over an indefinite time period, and $E_{i}$ be the energy in state $x_{i}$ . The system will have 2 contraints, the probabilities must add to 1, and the average energy, the internal energy, should equal $U$ .

\sum_{i = 1}^{N} p_{i} = 1

\sum_{i = 1}^{N} p_{i} E_{i} = U

We will assume that $S$ should be maximized.

S (p) = - k_{B} \sum_{i = 1}^{N} p_{i} \ln (p_{i})

We want to find the probability distribution $p = p_{1} \dots p_{n}$ that maximizes $S$ , while following the 2 constraints.

p_{max} = {argmax}_{P} S (p)

We can solve this system of equations with a Lagrange multiplier. First rewrite the 2 constraints like:

\sum_{i = 1}^{N} p_{i} - 1 = 0

and

\sum_{i = 1}^{N} p_{i} E_{i} - U = 0

Then make the Lagrange multiplier equation

L (p, λ_{1}, λ_{2}) = - k_{B} \sum_{i = 1}^{N} p_{i} \ln (p_{i}) + λ_{1} (\sum_{i = 1}^{N} p_{i} - 1) + λ_{2} (\sum_{i = 1}^{N} p_{i} E_{i} - U)

At the maximum the derivative with respect to any $p_{i}$ should be zero.

\frac{\partial L (p, λ_{1}, λ_{2})}{\partial p_{i}} = - k_{B} (\ln (p_{i}) - p_{i} \frac{1}{p_{i}}) + λ_{1} + λ_{2} E_{i}

Set the derivative to zero.

0 = - k_{B} (\ln (p_{i}) - p_{i} \frac{1}{p_{i}}) + λ_{1} + λ_{2} E_{i}

Then Solve for $p_{i}$ .

\ln (p_{i}) = \frac{- k_{B} + λ_{1} + λ_{2} E_{i}}{k_{B}}

Exponentiate both sides

p_{i} = e^{\frac{- k_{B} + λ_{1} + λ_{2} E_{i}}{k_{B}}}

Then split the exponent.

p_{i} = e^{- 1 + \frac{λ_{1}}{k_{B}}} e^{\frac{λ_{2} E_{i}}{k_{B}}}

Now we have to solve for the 2 multipliers. Start with the first constraint:

\sum_{i = 1}^{N} p_{i} = 1

And substitute $p_{i}$ .

\sum_{i = 1}^{N} e^{- 1 + \frac{λ_{1}}{k_{B}}} e^{\frac{λ_{2} E_{i}}{k_{B}}} = 1

The first term is independent of $i$ so it can be moved out of the sum.

e^{- 1 + \frac{λ_{1}}{k_{B}}} \sum_{i = 1}^{N} e^{\frac{λ_{2} E_{i}}{k_{B}}} = 1

Then divide both sides by the sum.

e^{- 1 + \frac{λ_{1}}{k_{B}}} = \frac{1}{\sum_{i = 1}^{N} e^{\frac{λ_{2} E_{i}}{k_{B}}}}

Now $e^{- 1 + \frac{λ_{1}}{k_{B}}}$ can be substituted.

p_{i} = \frac{e^{\frac{λ_{2} E_{i}}{k_{B}}}}{\sum_{i = 1}^{N} e^{\frac{λ_{2} E_{i}}{k_{B}}}}

Next solve to solve for $λ_{2}$ use the second constraint:

\sum_{i = 1}^{N} p_{i} E_{i} = U

Substitute in the equation for $p_{i}$ .

\sum_{i = 1}^{N} \frac{e^{\frac{λ_{2} E_{i}}{k_{B}}}}{\sum_{j = 1}^{N} e^{\frac{λ_{2} E_{j}}{k_{B}}}} E_{i} = U

Look at the equation for the maximum entropy:

S_{max} (U) = \max_{P \in P_{U}} S (P) = - k_{B} \sum_{i = 1}^{N} p_{i} \ln (p_{i}) = - \sum_{i = 1}^{N} \frac{e^{\frac{λ_{2} E_{i}}{k_{B}}}}{\sum_{j = 1}^{N} e^{\frac{λ_{2} E_{j}}{k_{B}}}} \ln (\frac{e^{\frac{λ_{2} E_{i}}{k_{B}}}}{\sum_{k = 1}^{N} e^{\frac{λ_{2} E_{k}}{k_{B}}}})

Then simplify

S_{max} = - k_{B} \sum_{i = 1}^{N} \frac{e^{\frac{λ_{2} E_{i}}{k_{B}}}}{\sum_{j = 1}^{N} e^{\frac{λ_{2} E_{j}}{k_{B}}}} \ln (\frac{e^{\frac{λ_{2} E_{i}}{k_{B}}}}{\sum_{k = 1}^{N} e^{\frac{λ_{2} E_{k}}{k_{B}}}})

Split the logarithm

S_{max} = - k_{B} \sum_{i = 1}^{N} \frac{e^{\frac{λ_{2} E_{i}}{k_{B}}}}{\sum_{j = 1}^{N} e^{\frac{λ_{2} E_{j}}{k_{B}}}} (\ln (e^{\frac{λ_{2} E_{i}}{k_{B}}}) - \ln (\sum_{k = 1}^{N} e^{\frac{λ_{2} E_{k}}{k_{B}}}))

Cancel the logarithm and exponent

S_{max} = - k_{B} \sum_{i = 1}^{N} \frac{e^{\frac{λ_{2} E_{i}}{k_{B}}}}{\sum_{j = 1}^{N} e^{\frac{λ_{2} E_{j}}{k_{B}}}} (\frac{λ_{2} E_{i}}{k_{B}} - \ln (\sum_{k = 1}^{N} e^{\frac{λ_{2} E_{k}}{k_{B}}}))

Split the summation.

S_{max} = - k_{B} \sum_{i = 1}^{N} \frac{e^{\frac{λ_{2} E_{i}}{k_{B}}}}{\sum_{j = 1}^{N} e^{\frac{λ_{2} E_{j}}{k_{B}}}} \ln (\sum_{k = 1}^{N} e^{\frac{λ_{2} E_{k}}{k_{B}}}) - k_{B} \sum_{i = 1}^{N} \frac{e^{\frac{λ_{2} E_{i}}{k_{B}}}}{\sum_{j = 1}^{N} e^{\frac{λ_{2} E_{j}}{k_{B}}}} (\frac{λ_{2} E_{i}}{k_{B}})

The second term in the first summation is independent so it can be moved out.

S_{max} = k_{B} \ln (\sum_{k = 1}^{N} e^{\frac{λ_{2} E_{k}}{k_{B}}}) \sum_{i = 1}^{N} \frac{e^{\frac{λ_{2} E_{i}}{k_{B}}}}{\sum_{j = 1}^{N} e^{\frac{λ_{2} E_{j}}{k_{B}}}} - k_{B} \sum_{i = 1}^{N} \frac{e^{\frac{λ_{2} E_{i}}{k_{B}}}}{\sum_{j = 1}^{N} e^{\frac{λ_{2} E_{j}}{k_{B}}}} (\frac{λ_{2} E_{i}}{k_{B}})

Then from the first constraint the entire first summation is just equal to 1.

S_{max} = k_{B} \ln (\sum_{k = 1}^{N} e^{\frac{λ_{2} E_{k}}{k_{B}}}) - k_{B} \sum_{i = 1}^{N} \frac{e^{\frac{λ_{2} E_{i}}{k_{B}}}}{\sum_{j = 1}^{N} e^{\frac{λ_{2} E_{j}}{k_{B}}}} (\frac{λ_{2} E_{i}}{k_{B}})

Move the $λ_{2}$ out of the summation.

S_{max} = k_{B} \ln (\sum_{k = 1}^{N} e^{\frac{λ_{2} E_{k}}{k_{B}}}) - k_{B} \frac{λ_{2}}{k_{B}} \sum_{i = 1}^{N} \frac{e^{\frac{λ_{2} E_{i}}{k_{B}}}}{\sum_{j = 1}^{N} e^{\frac{λ_{2} E_{j}}{k_{B}}}} E_{i}

Then the summation is just equal to $U$ .

S_{max} = k_{B} \ln (\sum_{k = 1}^{N} e^{\frac{λ_{2} E_{k}}{k_{B}}}) - λ_{2} U

Differentiate with respect to the internal energy.

\frac{\partial S_{max}}{\partial U} = k_{B} \frac{\partial \ln (\sum_{k = 1}^{N} e^{\frac{λ_{2} E_{k}}{k_{B}}})}{\partial U} - \frac{\partial (U λ_{2})}{\partial U}

Chain rule on the left and product rule on the right.

\frac{\partial S_{max}}{\partial U} = k_{B} (\frac{1}{\sum_{k = 1}^{N} e^{\frac{λ_{2} E_{k}}{k_{B}}}} \frac{\partial \sum_{k = 1}^{N} e^{\frac{λ_{2} E_{k}}{k_{B}}}}{\partial U}) - λ_{2} - U \frac{\partial λ_{2}}{\partial U}

The derivative operator distributes over addition.

\frac{\partial S_{max}}{\partial U} = k_{B} (\frac{1}{\sum_{k = 1}^{N} e^{\frac{λ_{2} E_{k}}{k_{B}}}} \sum_{k = 1}^{N} \frac{\partial e^{\frac{λ_{2} E_{k}}{k_{B}}}}{\partial U}) - λ_{2} - U \frac{\partial λ_{2}}{\partial U}

Chain rule again.

\frac{\partial S_{max}}{\partial U} = k_{B} (\frac{1}{\sum_{k = 1}^{N} e^{\frac{λ_{2} E_{k}}{k_{B}}}} \sum_{k = 1}^{N} e^{\frac{λ_{2} E_{k}}{k_{B}}} \frac{E_{k}}{k_{B}} \frac{\partial λ_{2}}{\partial U}) - λ_{2} - U \frac{\partial λ_{2}}{\partial U}

\frac{\partial S_{max}}{\partial U} = \frac{\partial λ_{2}}{\partial U} \frac{\sum_{k = 1}^{N} E_{k} e^{\frac{λ_{2} E_{k}}{k_{B}}}}{\sum_{k = 1}^{N} e^{\frac{λ_{2} E_{k}}{k_{B}}}} - λ_{2} - U \frac{\partial λ_{2}}{\partial U}

And the expression is just internal energy again.

\frac{\partial S_{max}}{\partial U} = \frac{\partial λ_{2}}{\partial U} U - λ_{2} - U \frac{\partial λ_{2}}{\partial U}

Those cancel out.

\frac{\partial S_{max}}{\partial U} = λ_{2}

So we have this expression which is the thermodynamic temperature.

λ_{2} = \frac{\partial S_{max}}{\partial U} = \frac{1}{T}

Because you define $S_{max}$ with $λ_{2}$ but then define $λ_{2}$ as the derivative of $S_{max}$ which creates a circular definition. For this part to be valid you must already know either the temperature (canonical ensemble) or the average energy.

Substituting, the full equation becomes

p_{i} = \frac{e^{\frac{1}{k_{B} T} E_{i}}}{\sum_{i = 1}^{N} e^{\frac{1}{k_{B} T} E_{i}}}