Entropy – Increased by Stirring, Decreased by Observation

1  Example and Simulation

2  Operating Instructions

• Click on the grid to raise or lower the veil. If you click on a row that is above the veil, that row (and all lower rows) will become veiled. If you click on a row that is below the veil, that row (and all higher rows) will become unveiled.
• The "Peek" button temporarily lowers the veil all the way, then raises it back to where it was. This allows you to discover where all the particles are. This is one way of making a zero-entropy state.
• The "Purify" button sweeps all the occupied states over to the left side of the grid. This is another way of making a zero-entropy state.
• The "Stirring: start" button causes automatic stirring, i.e. one stirring event every 50 milliseconds. The "stop" button puts a stop to this. The "once" button causes just one stirring event.

3  Principles

Conceptually, entropy is a measure of how much we don’t know about the system. See reference 1 for details.

Formally, entropy is defined in terms of probability; specifically:

where P_i is the probability of the ith microstate and the sum runs over all possible microstates of the system.

In the grid shown here, each site can have two possible states: occupied or unoccupied. The coloring shows the probability of occupation: white means unoccupied, i.e. 0% probability, while solid black (or solid red) means 100% probability, and shades of gray indicate intermediate probabilities.

The system is set up so that we can observe some of the sites ("above the veil") and not observe other sites ("below the veil").

• For each site above the veil, we can determine, just by looking, whether it it is occupied or not, so its probability of occupation is either 0% or 100%, never anything in between. Occupied sites above the veil are shown in red.

  In accordance with equation 1, sites above the veil make zero contribution to the total entropy. That’s because P_i log(1/¶_i) is zero when P_i=1, and also when P_i=0.

• For each site below the veil, things are much more interesting. We might or might not know very much about the site. If we have in the recent past temporarily lowered the veil and peeked at the site, we can remember whether it was occupied at that time, and if there hasn’t been very much stirring since then, the probability of occupation won’t have changed very much. On the other hand, stirring causes the particles to move around, gradually blurring out the probability for sites behind the veil.

  Sites below the veil contribute to the entropy in the usual way, in accordance with equation 1. If you remember where the veiled particles are (perhaps because of a recent peek), the entropy is zero, but stirring causes the entropy to grow.

Each stirring event has a source and a destination. These are chosen randomly, without regard to where the veil is.

• If the source and destination are both above the veil, a particle just hops from the source location to the destination location (provided the former was occupied and the latter was not already occupied – otherwise nothing happens).
• If the source is below the veil, we decrement the probability for each site below the veil, in proportion to how much probability-of-occupation was previously on that site. The total decrement (summed over all sites) is one unit.
• Similarly, if the destination is below the veil, we increment the probability for each site below the veil, in proportion to how much probability-of-vacancy was previously on that site. The total increment (summed over all sites) is one unit.

When the grid portrays the occupation of sites below the veil (black, white, or shades of gray) that does not mean you are "seeing through the veil". This part of the grid is not telling you anything you did not already know ... it is just helping you keep track of what you remember from the last time you peeked. In contrast, the part of the grid above the veil constantly provides complete up-to-date information about the occupancy of the exposed sites.

It is amusing to hit the Purify button, and then hit the Peek button a few times while the system is still evolving, before it reaches its maximum-entropy state.

As the saying goes, if you’ve see one two-state system, you’ve seen them all. So far, we have interpreted our model system in terms of particles hopping around on a lattice, such that each site was occupied or unoccupied. Therefore it was appropriate to measure the molar entropy (lower-case s) on a per-particle basis. However, we could equally well re-interpret it as a lattice of spin-1/2 objects, such that each site could be spin-up or spin-down. (This is relevant to some interesting applications, such as adiabatic demagnetization refrigerators.) The mathematics is the same in both cases. The only difference is that in the latter case, it would be more conventional to measure the molar entropy on a per-site basis. This is denoted by s′ in the caption to the grid. The distinction is purely a matter of convention; the total entropy (capital S) is what really matters, and it is the same in either interpretation.

4  Discussion

The main point of this exercise is to make clear the important distinction between mixing and probability. Entropy is defined in terms of probability. This definition is not going to change anytime soon. Mixing is not the same as probability. If you want a high-entropy state, it is usually necessary but never sufficient for the occupied sites and the vacant sites to be well mixed together. The crucial contribution to the entropy is not knowing where the particles are. If you know where they are, it doesn’t matter whether they are mixed or not ... as you can demonstrate with the "Peek" button:

• Peeking sets the entropy to zero, but does not unmix the particles.
• Purifying the system unmixes the particles, which is another way of setting the entropy to zero.
• For this system, the only way of adding entropy is by stirring. (However, if the veil is all the way down, you can stir until the cows come home and the entropy will remain zero.)
• Conversely, lack of mixing does not cause the entropy to decrease. If you turn off the mixing, the entropy does not change. It will not decrease unless you do something to find out where the particles are (e.g. purifying them or peeking at them).

To say the same thing another way: Some people labor under the misconception that entropy must somehow be a highly dynamic, active process, requiring constant mixing and re-mixing. There is no experimental or theoretical basis for believing this, and counterexamples abound. For example, suppose I hand you a chunk of glass (or if you want to be fancy, a chunk of spin glass). It has a very considerable amount of entropy that is frozen in, completely non-dynamic. The point is, you don’t know what microstate it’s in, and you can’t easily figure it out, so it really doesn’t matter whether the microstate is changing or not.

You can’t pretend that "all entropy is the entropy of mixing", because peeking at a highly-mixed system changes the entropy but does not change the degree of mixing ... not according to the usual definition of mixing. The Peek button behaves differently from the Purify button.

Specialists note: In the context of quantum statistical mechanics, when talking about density matrices, there is a concept of "mixed state" as opposed to "pure state", which is more-or-less directly related to entropy. This is a somewhat specialized, technical definition. It makes perfect sense to experts, but is not suitable for introducing non-experts to the idea of entropy. For example: A density matrix consisting of one electron in Pasadena and another electron in Boston would (almost certainly) be a mixed state, whereas a density matrix for two electrons sharing an sp-hybrid orbital in a single atom would (almost certainly) be a pure state. This rather dramatically conflicts with intuitive notions of mixing ordinary fluids.

Ordinary mixing refers to the distribution of particles over the 400 sites in the two-dimensional grid. The "mixed state" of a density matrix involves things that live in a 400-dimensional Hilbert space. (If you don’t know what this means, don’t worry about it.) Please don’t confuse mixing of particles with mixing of density matrices.

Entropy is defined in terms of the ensemble average of log(1/P_i). Get used to it.

5  References

1.

John Denker,
“Modern Thermodynamics”
www.av8n.com/physics/thermo-laws.htm