Fundamental assumption

The difference between these two approaches is in what is considered the fundamental assumption. In the Gibbs entropy approach we assumed that the entropy was a “nice” function of the probabilities of microstates, which gave us the Gibbs formula. From there, we could maximize the entropy to find the probabilities under some set of constraints.

The Boltzmann approach makes what is perhaps simpler assumption, which is that if only microstates with a given energy are permitted, then all of the microstates with that energy are equally probable. (This scenario with all microstates having the same energy is the microcanonical ensemble.) Thus the macrostate with the most corresponding microstates will be most probable macrostate. The number of microstates corresponding to a given macrostate is called the multiplicity \(g(E,V)\). In this approach, multiplicity (which did not show up last week!) becomes a fundamentally important quantity, since the macrostate with the highest multiplicity is the most probable macrostate.

Outline of the week

One or two topics per day:

1. Quick version showing the conclusions we will reach.
2. Finding the multiplicity of a paramagnet (Chapter 1).
3. (Probably skipped in class) Combining two non-interacting systems; defining temperature.

Quick version

This quick version will tell you all the essential physics results for the week, without proof. The beauty of statistical mechanics (whether following the text or using the information-theory approach of last week) is that you don't actually need to take on either faith or experiment the connection between the statistical theory and the empirical definitions used in thermodynamics.

Multiplicity of a paramagnet

So now the question becomes how to find the number of microstates that correspond to a given energy \(g(E)\). Once we have this in an analytically tractable form, we can everything else we might care for (with effort). This is essentially a counting problem, and much of what you need is introduced in Chapter 1. We will spend some class time going over one example of computing the multiplicity. Consider a paramagnetic system consisting of spin \(\frac12\) particles that can be either up or down. Each spin has a magnetic moment in the \(\hat z\) direction of \(\pm m\), and we are interested in the total magnetic moment \(\mu_{tot}\), which is the sum of all the individual magnetic moments. Note that the magnetization \(M\) used in electromagnetism is just the total magnetic moment of the material divided by its volume. \begin{align} M &\equiv \frac{\mu_{tot}}{V} \end{align}

Note

It is confusingly common to refer to the total magnetic moment as the magnetization. Given either a numerical value or an expression, it's usually easy to tell what you've got by checking the dimensions.

Small Group Question

Work out how many ways a system of 4 spins can have any possible magnetization of enumerating all the microstates corresponding to each magnetization.

Now find a mathematical expression that will tell you the multiplicity of a system with an even number \(N\) spins and just one \(\uparrow\) spin. Then find the multiplicity for two \(\uparrow\) spins, and for three \(\uparrow\) spins.

Now find a mathematical expression that will tell you the multiplicity of a system with an even number \(N\) spins and total magnetic moment \(\mu_{tot}=2sm\) where \(s\) is an integer. We call \(s\) the spin excess, since \(N_\uparrow = \frac12N + s\). Alternatively, you could write your expression in terms of the number of up spins \(N_\uparrow\) and the number of down spins \(N_\downarrow\).

Answer

We can enumerate all spin microstates:

\(\mu_{tot}=-4m\)

\(\downarrow\downarrow\downarrow\downarrow\) g=1

\(\mu_{tot}=-2m\)

\(\downarrow\downarrow\downarrow\uparrow\) \(\downarrow\downarrow\uparrow\downarrow\) \(\downarrow\uparrow\downarrow\downarrow\) \(\uparrow\downarrow\downarrow\downarrow\) g=4

\(\mu_{tot}=0\)

\(\downarrow\downarrow\uparrow\uparrow\) \(\downarrow\uparrow\uparrow\downarrow\) \(\uparrow\uparrow\downarrow\downarrow\) \(\uparrow\downarrow\downarrow\uparrow\) \(\uparrow\downarrow\uparrow\downarrow\) \(\downarrow\uparrow\downarrow\uparrow\) g=6

\(\mu_{tot}=2m\)

\(\uparrow\uparrow\uparrow\downarrow\) \(\uparrow\uparrow\downarrow\uparrow\) \(\uparrow\downarrow\uparrow\uparrow\) \(\downarrow\uparrow\uparrow\uparrow\) g=4

\(\mu_{tot}=4m\)

\(\uparrow\uparrow\uparrow\uparrow\) g=1

To generalize this to \(g(N,s)\), we need to come up with a systematic way to count the states that have the same spin excess \(s\). Clearly if \(s=\pm N/2\), \(g=1\), since that means that all the spins are pointed the same way, and there is only one way to do that. \begin{align} g(N,s=\pm \frac12N) &= 1 \end{align} Now if we have just one spin going the other way, there are going to be \(N\) ways we could manage that: \begin{align} g\left(N,s=\pm \left(\frac12N-1\right)\right) &= N \end{align} Now when we go to flip it so we have two spins up, there will be \(N-1\) ways to flip the second spin. But then, when we do this we will end up counting every possibility twice, which means that we will need to divide by two. \begin{align} g\left(N,s=\pm \left(\frac12N-2\right)\right) &= N(N-1)/2 \end{align} When we get to adding the third \(\uparrow\) spin, we'll have \(N-2\) spins to flip. But now we have to be even more careful, since for the same three up-spins, we have several ways to reach that microstate. In fact, we will need to divide by \(6\), or \(3\times 2\) to get the correct answer (as we can check for our four-spin example). \begin{align} g\left(N,s=\pm \left(\frac12N-3\right)\right) &= \frac{N(N-1)(N-2)}{3!} \end{align} At this stage we can start to see the pattern, which comes out to \begin{align} g\left(N,s\right) &= \frac{N!}{\left(\frac12 N + s\right)!\left(\frac12N -s\right)!} \\ &= \frac{N!}{N_\uparrow!N_\downarrow!} \end{align}

Entropy of our spins

I'm going to use a different approach than the text to find the entropy of this spin system when there are many spins and the spin excess is relatively small. \begin{align} S &= k\ln g\left(N,s\right) \\ &= k\ln\left(\frac{N!}{\left(\tfrac12 N + s\right)!\left(\tfrac12N -s\right)!}\right) \\ &= k\ln\left(\frac{N!}{N_\uparrow!N_\downarrow!}\right) \\ &= k\ln\left(\frac{N!}{\left(h + s\right)!\left(h -s\right)!}\right) \end{align} At this point I'm going to define for convenience \(h\equiv \tfrac12 N\), just to avoid writing so many \(\tfrac12\). I'm also going to focus on the \(s\) dependence of the entropy. \begin{align} \frac{S}{k} &= \ln\left(N!\right) - \ln\left(N_\uparrow!\right) - \ln\left(N_\downarrow!\right) \\ &= \ln N! - \ln(h+s)! - \ln(h-s)! \\ &= \ln N! - \sum_{i=1}^{h+s} \ln i - \sum_{i=1}^{h-s} \ln i \end{align} At the last step, I wrote the log of the factorial as a sum of logs. This is still looking pretty hairy. So let's now consider the difference between the entropy with \(s\) and the entropy when \(s=0\) (which I will call here \(S_0\) for compactness and convenience). \begin{align} \frac{S(s)-S_0}{k_B} &= - \sum_{i=1}^{h+s} \ln i - \sum_{i=1}^{h-s} \ln i + \sum_{i=1}^{h} \ln i + \sum_{i=1}^{h} \ln i \\ &= -\sum_{i=h+1}^{h+s} \ln i + \sum_{j=h-s+1}^{h} \ln j \end{align} where I have changed the sums to account for the difference between the sums with \(s\) and those without. At this stage, our indices are starting to feel a little inconvenient given the short range we are summing over, so let's redefine our index ov summation so the sums will run up to \(s\). In preparation for this, at the last step, I renamed one of my dummy indexes. \begin{align} i &= h + k & j &= h + 1 - k \end{align} With these indexes, each sum can go from \(k=1\) to \(k=s\), which will enable us to combine our sums into one. \begin{align} \frac{S-S_0}{k} &= -\sum_{k=1}^{s} \ln(h+ k) + \sum_{k=1}^{s} \ln (h+1-k) \\ &= \sum_{k=1}^{s} \left(\ln (h+1-k) - \ln(h+ k)\right) \end{align} At this point, if you're anything like me, you're thinking “I could turn that difference of logs into a log of a ratio!” Sadly, this doesn't turn out to help us. Instead, we are going to start trying to get the \(h\) out of the way in preparation for taking the limit as \(s\ll h\). \begin{align} \frac{S-S_0}{k} &= \sum_{k=1}^{s} \ln h + \ln\left(1-\frac{k-1}{h}\right) - \ln h - \ln\left(1+ \frac{k}{h}\right) \\ &= \sum_{k=1}^{s} \left(\ln\left(1-\frac{k-1}{h}\right) - \ln\left(1+ \frac{k}{h}\right)\right) \end{align} It is now time to make our first approximation: we assume \(s\ll N\), which means that \(s\ll h\). That enables us to simplify these logarithms drastically! \(\ddot\smile\) \begin{align} \frac{S-S_0}{k} &\approx \sum_{k=1}^{s} \left(-\frac{k-1}{h} - \frac{k}{h}\right) \\ &= -\frac2{h}\sum_{k=1}^{s} \left(k-\tfrac12\right) \\ &= -\frac4{N}\sum_{k=1}^{s} \left(k-\tfrac12\right) \end{align}

Now we have this sum to solve. You can find this sum either geometrically or with calculus. The calculus involves turning the sum into an integral. As you can see in the figure, the integral \begin{align} \int_0^s x dx = \tfrac12 s^2 \end{align} has the same value as the sum, since the area under the orange curve (which is the sum) is equal to the area under the blue curve (which is the integral).

The geometric way to solve this looks visually very much the same as the integral picture, but instead of computing the area from the straight line, we cut the stair-step area “half” and fit the two pieces together such that they form a rectangle with width \(s/2\) and height \(s\).

Taken together, this tells us that when \(s\ll N\) \begin{align} S(N,s) &\approx S(N,s=0) - k\frac{4}{N}\frac{s^2}{2} \\ &= S(N,s=0) - k\frac{2s^2}{N} \end{align} This means that the multiplicity is gaussian: \begin{align} S &= k \ln g \\ g(N,s) &= e^{\frac{S(N,s)}{k}} \\ &= e^{\frac{S(N,s=0)}{k} - \frac{2s^2}{N}} \\ &= g(N,s=0)e^{-\frac{2s^2}{N}} \end{align} Thus the multiplicity (and thus probability) is peaked at \(s=0\) as a gaussian with width \(\sim\sqrt{N}\). This tells us that the width of the peak increases as we increase \(N\). However, the excess spin per particle decreases as \(\sim\frac{1}{\sqrt{N}}\). So that means that our fractional polarization becomes far more sharply peaked as we increase \(N\).

Thermal contact (probably skipped in class)

Suppose we put two systems in contact with one another. This means that energy can flow from one system to the other. We assume, however, that the contact between the two systems is weak enough that their energy eigenstates are unaffected. This is a bit of a contradiction you'll need to get used to: we treat our systems as non-interacting, but assume there is some energy transfer between them. The reasoning is that the interaction between them is very small, so that we can treat each system separately, but energy can still flow.

We ask the question: “How much energy will each system end up with after we wait for things to settle down?” The answer to this question is that energy will settle down in the way that maximizes the number of microstates.

Let us consider two simple systems: a 2-spin paramagnet, and a 4-spin paramagnet.

System \(A\)

A system of 3 spins each with energy \(\pm 1\). This system has the following multiplicity found from Pascal's triangle: \begin{equation*}\small \begin{array}{cccccccccc} 1 \\ 1 & 1 \\ 1 & 2 & 1 \\ 1 & 3 & 3 & 1 \\ \hline -3 & -1 & 1 & 3 \end{array} \end{equation*}

System \(B\)

A system of 4 spins each with energy \(\pm 1\). This system has the following multiplicity found from Pascal's triangle: \begin{equation*}\small \begin{array}{ccccccccccc} 1 \\ 1 & 1 \\ 1 & 2 & 1 \\ 1 & 3 & 3 & 1 \\ 1 & 4 & 6 & 4 & 1 \\ \hline -4 & -2 & 0 & 2 & 4 \end{array} \end{equation*}

Question

What is the total number of microstates when you consider systems \(A\) and \(B\) together as a combined system? Answer

We need to multiply the numbers of microstates for each system separately, because for each microstate of \(A\), it is possible to have \(B\) be in any of its microstates. So the total is \(2^32^4 = 128\).

Since we have two separate systems here, it is meaningful to ask what the probability is for system \(A\) to have energy \(E_A\), given that the combined system has energy \(E_{AB}\).

Small group question

What is the multiplicity of the combined system if the energy is 3, i.e. \(g_{AB}(E_{AB}=3)\)?

Answer

To solve this, we just need to multiply the multiplicities of the two systems and add up all the energy possibilities that total 3: \begin{align} g_{AB}(E_{AB}=0) &= g_A(-1)g_B(4) + g_A(1)g_B(2) + g_A(3)g_B(0) \\ &= 3\cdot 1 + 3\cdot 4 + 1\cdot 6 \\ &= 21 \end{align}

Small group question

What is the probability that system \(A\) has energy 1, if the combined energy is 3?

Answer

To solve this, we just need to multiply the multiplicities of the two systems, which we already found and divide by the total number of microstates: \begin{align} P(E_A=1|E_{AB}=3) &= \frac{g_A(1)g_B(2)}{g_{AB}(3)} \\ &= \frac{3\cdot 4}{21} \\ &= \frac{4}{7} \end{align} which shows that this is the most probable distribution of energy between the two subsystems.

Given that these two systems are able to exchange energy, they ought to have the same temperature. To find the most probable energy partition between the two systems, we need to find the partition that maximizes the multiplicity of the combined system: \begin{align} g_{AB}(E_A) &= g_A(E_A)g_B(E_{AB}-E_A) \\ 0 &= \frac{d g_{AB}}{d E_A} \\ &= g_A'g_B - g_B' g_A \\ \frac{g_A'}{g_A} &= \frac{g_B'}{g_B} \\ \frac{1}{g_A(E_A)} \frac{\partial g_A(E_A)}{\partial E_A} &= \frac{1}{g_B(E_B)} \frac{\partial g_B(E_B)}{\partial E_B} \end{align} This tells us that the “thing that becomes equal” when the two systems are in thermal contact is this strange ratio of the derivative of the multiplicity with respect to energy divided by the multiplicity itself. You may be able to recognize this as what is called a logarithmic derivative. \begin{align} \frac{\partial}{\partial E_A}\ln(g_A(E_A)) &= \frac{1}{g_A(E_A)} \frac{\partial g_A(E_A)}{\partial E_A} \end{align} thus we can conclude that when two systems are in thermal contact, the thing that equalizes is \begin{align} \beta &\equiv \left(\frac{\partial \ln g}{\partial E}\right)_V \end{align} At this stage, we haven't shown that \(\beta=\frac1{kT}\), but we have shown that it should be a function of \(T\), since \(T\) is also a thing that is equalized when two systems are in thermal contact.

By dimensional reasoning, you can recognize that this could be \(\frac1{kT}\), and we're just going to leave this at that.