(1) The Probability Concept, its Usefulness in the Past, its Present
Necessity, and its Universality.
An indication of its essential value in this physical discussion is evidenced by the fact that we have almost unwittingly been forced to constantly refer to it in all of our preliminaries. But when this concept is first broached to a student, he feels about it like the "man in the street"; it is by the latter regarded as a matter of chance and hence of uncertainty and unreliability; moreover, the latter knows in a vague way that the subject has to do with averages, that it is often of a statistical nature, and knows that statistics in general are widely distrusted. The student is at first likely to share these views with said man in the street, and at best feels that its introduction is of remote interest, far fetched, and tends to hide and dissipate the kernel of the matter. The student must disabuse himself of these false notions by reflecting how much there is in Nature that is spontaneous, in other words, how many events there are in which there is a passage from a less probable to a more probable condition and that he cannot afford to despise or ignore a Calculus which measures these changes as exactly as possible.
In this connection BOLTZMANN says: (W. S. B. d. Akad. d. Wiss., Vol. LXVI, B 1872, p. 275).
"The mechanical theory of heat assumes that the molecules of gases are in no way at rest but possess the liveliest sort of motion, therefore, even when a body does not change its state, every one of its molecules is constantly altering its condition of motion and the different molecules likewise simultaneously exist side by side in most different conditions. It is solely due to the fact that we always get the same average values, even when the most irregular occurrences take place under the same circumstances, that we can explain why we recognize perfectly definite laws in warm bodies. For the molecules of the body are so numerous and their motions so swift that indeed we do not perceive aught but these average values. We might compare the regularity of these average values with those furnished by general statistics which, to be sure, are likewise derived from occurrences which are also conditioned by the wholly incalculable co-operation of the most manifold external circumstances. The molecules are as it were like so many individuals having the most different kinds of motion, and it is only because the number of those which on the average possess the same sort of motion is a constant one that the properties of the gas remain unchanged. The determination of the average values is the task of the Calculus of Probabilities. The problems of the mechanical theory of heat are therefore problems in this calculus. It would, however, be a mistake to think any uncertainty is attached to the theory of heat because the theorems of probability are applied. One must not confuse an imperfectly proved proposition (whose truth is consequently doubtful) with a completely established theorem of the Calculus of Probabilities; the latter represents, like the result of every other calculus, a necessary consequence of certain premises, and if these are correct the result is confirmed by experience, provided a sufficient number of cases has been observed, which will always be the case with Heat because of the enormous number of molecules in a body."
To become more specific we will mention some of the problems to which the Theory of Probabilities has been profitably applied. In business to life and fire insurance; in engineering to reducing the inevitable errors of observations by the Method of Least Squares; and in physics to the determination of Maxwell's Law of the distribution of velocities. The results thus obtained are universally trusted and accepted by experts. Why then should this Calculus not be applicable to the more general natural events?
In this connection consider some of its good points: (a) It eliminates from a problem the accidental elements if the latter are sufficiently numerous; (b) it deals legitimately with averages; (c) it involves combination considerations other than averages; (d) it is available for non-mechanical as well as mechanical occurrences and thus (e) has a capacity for covering the whole range of natural events, giving it a character of universality which is now its most valuable asset.
As an example of this we may instance BOLTZMANN'S deservedly famous H-theorem, which establishes the one-sidedness of all natural events.[7] Concerning it, this master in mathematical physics says:
"It can only be deduced from the laws of probability that, if the initial state is not especially arranged for a certain purpose, the probability that decreases is always greater than that it increases. In this connection we may add that BOLTZMANN looked forward to a time, "when the fundamental equations for the motion of individual molecules will prove to be merely approximate formulas, which give average values which, according to the Theory of Probabilities, result from the co-operation of very many independently moving individuals constituting the surrounding medium, for example, in meteorology the laws will refer only to average values deduced by the Theory of Probabilities from a long series of observations. These individuals must of course be so numerous and act so promptly that the correct average values will obtain in millionths of a second."
To further strengthen our faith we may point out that the probability method has been successfully used to determine unique results from complicated conditions and has been employed for the general treatment of problems. In the case before us it has solved the entropy puzzle which has exercised physicists, as well as engineers, for decades, and it has thereby emancipated the Second Law from all anthropomorphism, from all dependence on human experimental skill. When we take the broadest possible view of its character, this Calculus enables us to read the present riddle of our universe, namely, why it is in its present improbable state. We have therefore in this Calculus an engine for investigation which is of great power and is likely to play a large part in the future in the ascertainment of physical truth. Of course it must then be in the hands of masters. It is they and they alone who can properly and adequately interpret such a physical problem as the one before us. In scientific work our last court of appeal must be Nature, and we therefore say: The best justification for the use of the Theory of Probabilities in our problem is that its results are in such complete accord with the facts.
In dealing with this physical engine of investigation, we must again call attention to some of the features of haphazard necessary for its legitimate application. Of course the statement of these features will vary with the mechanical or non-mechanical character of the problem to which it is applied. As we are here dealing mainly with the former, we will limit ourselves to its features: (a) The elements dealt with must be very numerous, strictly speaking, infinite; (b) as a phase of (a) we may say also that when we speak of the probability of a state we express the thought that it can be realized in many different ways; (c) when we speak of the relative directions of a pair of molecules all possible directions must be considered; (d) we must so weight the elements say, in (a), (b), and (c) that they are equally likely; (e) every one of the entering elements must possess constituents of which each individual is independent of every other; for instance, (f) in a gas the place where a molecule collided must be independent of the place where it collided before. In our physical problem all of these features are not always realized; for instance, the number of particles of gas are only finite instead of being infinite; again, all relative velocities after collision of a pair of molecules are not equally likely; BOLTZMANN and BURBURY provide for these shortcomings by very truly asserting that in actual cases we are not dealing with isolated systems, that the surrounding walls are not impervious to external influences, and that the latter come at haphazard without regard to internal state of the system at the time, thus renewing and maintaining the desired state of haphazard.
Methods. This Calculus works largely by the determination of averages and its results must be interpreted accordingly. Moreover, for the present we will take a popular, practical view of these results and consider a very great improbability as equivalent to an impossibility. Numerical computations are essential in most uses of this Calculus, but here they will be entirely omitted.
[7]The H-theorem considers a process (consisting of a number of separate, reversible processes) which is irreversible in the aggregate.
(2) What is Meant by the Probability of a State? Example
To come back to the matter in hand we will now show what is here meant by the probability of any state.
When we speak of the probability of a particular "elementar-ungeordnete" state, we thereby imply that this state may be variously realized. For every state (which contains many like independent constituents) corresponds to a certain "distribution," namely, a distribution among the gas molecules of the location co-ordinates and of the velocity components. But such a distribution is a permutation problem, is always an assignment of one set of like elements (co-ordinates, velocity components) to a different set of like elements (molecules). So long as only a particular state is kept in view, it is of consequence as to how many elements of the two sets are thus interchangeably assigned to each other and not at all as to which individual elements of the one set are assigned to particular individual elements of the other set.[8] Then a particular state may be realized by a great number of assignments individually differing from one another, but all equally likely to occur.[9] If with PLANCK we call such an assignment a "complexion,"[10] we may now say that in general a particular state contains a large number of different, but equally likely, complexions. This number, i.e., the number of the complexions included in a given state can now be defined as the probability of the state.[11] Let us present the matter in still another form. BOLTZMANN derives the expression for magnitude of the probability by at once distinguishing between a state of a considered system and the complexion of the considered system. A state of the system is determined by the law of locus and velocity distribution, i.e., by a statement of the number of particles which lie in each elementary district of space and the number of particles which lie in each elementary velocity realm, assuming that among themselves these districts and realms are alike and each such infinitesimal element still harbors very many particles. Accordingly a particular state of the system embraces a very large number of complexions. For if any two particles belonging to different regions swap their co-ordinates and velocities, we get thereby a new complexion, but still the same state. Now BOLTZMANN assumes all complexions to be equally probable and therefore the number of complexions included in a particular state furnishes at the same time the numerical value for the Probability of the state in question. Illustration taken from the simultaneous throwing of two, ordinary, cubical dice. Suppose that the sum is to be 4 for each throw, then this can be realized by the following three complexions:
First cube shows 1, the second cube shows 3;
First cube shows 2, the second cube shows 2;
First cube shows 3, the second cube shows 1.
The requirement that the sum on the two cubes shall be 2, however, involves but one complexion. Under the circumstances therefore the probability of throwing the sum 4 is three times as great as throwing the sum 2.
In closing this part of our presentation, we may make what is now an almost obvious remark. The long-lasting difficulty in giving a physical meaning to entropy and the Second Law is due to the fact of its intimate dependence on considerations of probability. It is only quite recently that such considerations have attained the dignity of a great working principle in the domain of Physics.
[9]LIOUVILLE'S theorem is the criterion for the equal possibility or equal probability of different state distributions.
[10]A happy term, but one not in vogue among English-speaking physicists.
[11]The identity of entropy with the logarithm of this state of probability is established by showing that both are equal to the same expression. It seems an easy step from this derivation to BOLTZMANN'S definition of entropy as the "measure of the disorder of the motions in a system of mass points."