VI. On The Presumed Extensive Character of Entropy
Most thermodynamic variables appear naturally to have either an extensive or intensive character; that is, they scale with the size of the system (V, N, E, ¼), or they do not (T, n = N/V, P, ¼). Because S is presumed additive - S = S1+S2 for a combination of two subsystems - one might expect entropy to be extensive. 1 But additivity is not guaranteed in general, as exemplified by any inhomogeneous system, and particularly those whose particles interact under long-range forces (e.g., ionic crystals, or spin systems in which the Heisenberg Hamiltonian is not restricted to nearest neighbors): owing to long-range correlations one generally has S < S1+S2, while the Clausius definition (6-2) (below) remains valid as long as N is constant. An extreme example due to Nelson was also noted earlier following Eq.(3-34), and some further relevant discussion will be given in Section IX in connection with gravitational interactions.
Many sources, tacitly if not explicitly, equate additivity and extensivity;
this, too, can lead to trouble. As in Section II, we shall define a function
of extensive variables {Yi} as extensive if it satisfies
the scaling law
|
|
Generally the terms `intensive' and `extensive' are somewhat ambiguous. This distinction is in part an anthropomorphic one, because it may depend on the particular kind of subdivision we choose to imagine. For example, a volume of air may be imagined to consist of a number of smaller contiguous volume elements. With this subdivision, the pressure is the same in all subsystems, and is therefore intensive; while the volume is additive and therefore extensive. But we may equally well regard the volume of air as composed of its constituent nitrogen and oxygen subsystems, and with this kind of subdivision the volume is the same in all subsystems, while the pressure is the sum of the partial pressures of its constituents; it appears that the roles of `intensive' and `extensive' have been interchanged.
Note that this ambiguity cannot be removed by requiring that we consider only spatial subdivisions, such that each subsystem has the same local composition. For, consider a stressed elastic solid, such as a stretched rubber band. If we imagine the rubber band as divided, conceptually, into small subsystems by passing planes through it normal to its axis, then the tension is the same in all subsystems, while the elongation is additive. But if the dividing planes are parallel to the axis, the elongation is the same in all subsystems, while the tension is additive; the roles of `extensive' and `intensive' are interchanged merely by imagining a different kind of spatial subdivision.
In spite of the fundamental ambiguity of the usual definitions, the notions of extensive and intensive variables are congenial, and in practice we seem to have no difficulty in deciding which quantities should be considered intensive.
But the issue for entropy goes beyond these considerations, as is seen
by re-examining the so-called Gibbs paradox. In his great work on the equilibrium
of heterogeneous systems (1875-1878) Gibbs briefly discussed the entropy
of mixing of two ideal gases and, since his statistical mechanics was not
yet fully developed, he was discussing only differences in the experimental
entropy defined by Clausius:
|
|
||||||||||||||||||||||||||||||||||||||||
|
Following Gibbs, we consider two such ideal gases with equal densities
n = N1/V1 = N2/V2
at the same constant Tand P, separated by a diaphragm in
a volume V = V1+V2 (Fig.2).
The diaphragm is removed and eventually a new equilibrium state is reached
with N = N1+N2 particles in
volume V = V1+V2, while the
temperature, pressure and total energy remain unchanged. If the two gases
are materially different the change in entropy is then 2
|
||||||||||||||
In the ensuing discussion Gibbs notes that when two unlike gases mix and the entropy increases there is an implication that the gases in principle could be separated again, bringing them back to their original macroscopic states at the expense of making changes in the external environment. This does not mean that each particle would be restored to its original position, however, but only that the initial macrostates would be restored; it would only be required to return each particle originally in V1 to that volume, and likewise for V2. Doing so, of course, would be very difficult to carry out by manipulating macroscopic variables alone.
He also sees that the situation for two identical gases is entirely different, and when we say that two identical gases mix with no change in entropy we do not mean that molecules originally in V1 can be restored to V1 without external changes. One cannot distinguish these two thermodynamic states in any event, for there has been no change in the thermodynamic state of the system. Indeed, restoration occurs by simply reinserting the diaphragm, so that the entropy change is zero. He clearly considers mixtures of like and unlike gases to be on ``a different footing'', indicating that comparing the two situations is not even conceptually possible.
But Gibbs also notes that for different gases there is virtually no limit to the resemblance they might have to one another, differing only in some tiny physical detail, and still lead to an entropy of mixing (6-5), independent of those details. Many writers on this problem have seen its essence to be this apparent discontinuity in DS as the physical characteristics of the two gases are continuously made the same. For example, Denbigh and Redhead (1989) have argued that there should be at least some diminution in DS as these changes occur. Aside from the fact that one simply can't do this with only macroscopic control, the argument misses the point entirely. The discontinuity here lies with what we mean by `restoring the original state,' and for like gases the phrase is meaningless. In one case information is lost upon mixing, but not in the other. Only if S is interpreted as mechanical is this paradoxical.
What Gibbs sees clearly and emphasizes strongly is that S is both macroscopic and statistical in nature. For ideal gases the equation of state PV = NkT is itself a macroscopic description that is the same for all ideal gases and implies no macroscopic differences for other thermodynamic functions. ``In such respects, entropy stands strongly contrasted with energy.'' Microscopic differences, such as weak interactions among particles, make no contribution at this level, and hence we should not expect to see the entropy of mixing depend on the detailed microscopic characteristics of the gases. Of course, if the constituents of the two unlike gases interact strongly enough with one another to have a macroscopic impact, then the entropy of mixing will not be expected to have the form (6-5) in any event. Finally, by way of further emphasis on the statistical character of S, he notes that there is still a small probability that demixing could occur spontaneously, for either like or unlike gases; there is no difference between the two cases in this respect. So, to put words into Gibbs' mouth, what's the problem?
Clearly there is no problem as long as we are discussing the thermodynamic
entropy of Clausius, Eq.(6-2). But a difficulty does arise when we move
to statistical mechanics and the quantum mechanical canonical ensemble,
in which the theoretical maximum entropy that is taken to be equivalent
to (61) is given by
|
The conventional resolution of this begins by recognizing that the canonical
Stheor in (6-6) is not an extensive quantity, but can
be made so by further adjusting the arbitrary constant. Replacement of
the particle numbers in (6-5) with their factorials and reference to Stirling's
formula suggests that all will be well if we subtract from Stheor
a term klnN!, which leads to
|
|||||||||||||||||||||
Now when the two gases are mixed and DStheor is calculated using (6-7) we find the expected behavior: (DS)1 ¹ 2 ¹ 0, and (DS)1 = 2 = 0, and the paradox is resolved. 3 It was Gibbs who actually noticed the need for this N! correction and uses it in his statistical mechanics book, following a discussion of classical indistinguishability. In SM (p.188) he introduces ``the supposition that phases are not altered by the exchange of places between similar particles.'' (This observation is pursued further by Pesic, 1991.)
From where, though, comes this term in lnN!? As is well known, the conventional wisdom is that its origin lies with quantum statistical mechanics and the notion of identical particles. When N-particle symmetrized state vectors are properly normalized so that the trace of the density matrix is unity, there is an overall factor of N! dividing Tre-bH, and this remains even when the particles are not identical. This is sometimes referred to as ``correct Boltzmann counting'', and the resulting formalism referred to as Boltzmann statistics. The latter is taken to be the correct classical limit, so that classical statistics itself is considered to be incorrect. Gibbs, of course, had no way of comprehending this situation, though the specific heat problem had alerted him to the fact that something was wrong (hence his introduction of the factor).
But does this resolution via quantum statistics and indistinguishability really get to the heart of the matter? Given that there can exist systems for which the thermodynamic entropy is by no means extensive, is this property of Stheor the entire story? There are good reasons to think that the answers to these questions may not be as straightforward as previously thought. Indeed, Ehrenfest and Trkal (1920) had much earlier questioned the presumption that entropy must be an extensive quantity and strongly criticized ad hoc additions of the term klnN!.
Return to the thermodynamic entropy of Clausius in the form (6-2) and note that it, too, is not extensive, except in some special cases. 4 Only T and V are varied on the reversible path in (6-3), and hence this equation can only determine the dependence of S on those variables. In fact, if N actually varied on that path then (6-2) would have to contain an additional `entropy convection' term ò(m/T) dN, as is also suggested in Eq.(6-7). Ehrenfest and Trkal (1920) had already appreciated these points explicitly and noted that one must really introduce a reversible process in which the number of molecules changes; they were properly sceptical about the possibility of doing this.
In his lectures circa 1952 Pauli (1973) seems to be one of the
first to have appreciated this point explicitly and noted that the entropy
of Eq.(6-4) should be replaced by
|
|
|
|
The single arbitrary constant f(1) in this last expression is
essentially the chemical constant, but it is not determined by any of the
foregoing expressions. The best that can be done by dimensional analysis
alone is to rewrite it as
|
The entropies of both quantum and Boltzmann statistics are usually defined in analogy with the Clausius definition (6-2), and so are both subject to this same kind of Pauli analysis. That is, if the statistical forms are to be extensive they too must conform to the scaling law (6-1); hence the variational principles from which they are derived also allow for an arbitrary function g(N) in their expression, which is basically that of Eq.(6-10). At this level experiment seems to indicate that the associated true constant g(1) is unity, and the effect of g(N) is the same as that of the lnN! term. Only when we consider processes that vary with N, as in the grand canonical ensemble, can we hope to obtain a proper definition of entropy in that case, and even then the experimental definition of Clausius must be completed in an appropriate manner. In addition, correct entropy expressions should allow for surface effects in a finite system, and that of Clausius does not.
The effect of these musings is to convince us that a proper definition
of entropy should at bottom be theoretical, and not based on the incomplete
expression of Clausius, Eq.(6-2). As Jaynes (1992) has pointed out in his
discussion of this problem, it would be very difficult for any phenomenological
theory based on a finite number of observations to provide a complete and
general definition of entropy. It is not an empirical question, but a conceptual
one. There are different ways one might approach this, one of which is
purely macroscopic, such as in the manner of Lieb and Yngvason discussed
earlier. For reasons mentioned there, our view is that we should strive
for a statistical theory based soundly on the principles of probability
theory, in which Stheor is a function of the macrovariables
employed to define the macrostate and satisfies a variational principle:
it is the upper bound of -kTr(rlogr)
over all density matrices in agreement with those macrovariables. This
result will automatically provide all the extra terms needed to analyze
any system described by that macrostate, and will agree with both the Clausius
and Pauli expressions in those cases where they are appropriate. If processes
in which N varies are included in the macroscopic description, as
in the grand canonical ensemble, guidance for what should be included in
the experimental entropy is immediate; the matter of extensivity takes
care of itself. 5
Questions about how to measure entropy changes are addressed by the macroscopic
thermodynamic equations themselves. We have outlined just such an approach
to the statistical theory in Section II above, while a much more detailed
description can be found in Grandy (1987).
2When Gibbs wrote on this, Boltzmann's constant had not yet been introduced, so he worked with moles and the gas constant R.
3This result for like gases suggests that isentropic processes should be characterized by the macroscopic condition Wf = Wi.
4DS will be proportional to N whenever the heat capacity is proportional to N and the pressure depends on N and V only in the ratio N/V. This is so for the ideal gas, but may not be so in other systems.
5For example, in a photon gas S(T,V,E) is automatically homogeneous of degree 1, presumably because particle number is not conserved.
