The ``true value'' of the mean could be defined by flipping the coin n times and conceiving of n going to infinity. A more convenient definition of ``true value'' is that the experiment could be conceived of as having been done separately under identical conditions by an infinite number of people (an ensemble). Such an artifice will enable us to define a time-variable mean for coins which change with time.
The utility of the concept of an ensemble is often subjected to serious attack both from the point of view of the theoretical foundations of statistics and from the point of view of experimentalists applying the techniques of statistics. Nonetheless a great body of geophysical literature uses the artifice of assuming the existence of an unobservable ensemble. The advocates of using ensembles (the Gibbsians) have the advantage over their adversaries (the Bayesians) in that their mathematics is more tractable (and more explainable). So, let us begin!
A conceptual average over the ensemble, called an expectation, is denoted by the symbol E. The index for summation over the ensemble is never shown explicitly; every random variable is presumed to have one. Thus, the true mean at time t may be defined as
Likewise, we may be interested in a property of xt called its variance which is a measure of variability about the mean defined by
Now let xt be a time series made up from (identically distributed, independently chosen) random numbers in such a way that m and do not depend on time. Suppose we have a sample of n points of xt and are trying to determine the value of m. We could make an estimate of the mean m with the formula
Our objective in this section is to determine how far the estimated mean is likely to be from the true mean m for a sample of length n. One possible definition of this excursion is
When one is trying to estimate the mean of a random series which has a time-variable mean, one faces a basic dilemma. If one includes a lot of numbers in the sum to get small, then m may be changing while one is trying to measure it. In contrast, measured from a short sample of the series might deviate greatly from the true m (defined by an infinite sum over the ensemble at any point in time). This is the basic dilemma faced by a stockbroker when a client tells him, ``Since the market fluctuates a lot I'd like you to sell my stock sometime when the price is above the mean selling price.''
If we imagine that a time series is sampled every seconds and we let denote the length of the sample then (30) may be written as
In considering other sets of weights one may take a definition of which is more physically sensible than times the number of weights. For example, if the weights wt are given by a sampled gaussian function as shown in Figure 2 then could be taken as the separation of half-amplitude points, 1/e points, the time span which includes 95 percent of the area, or it could be given many other ``sensible'' interpretations. Given a little slop in the definition of and ,it is clear that the inequality of (32) is not to be strictly applied.
Figure 2 Binomial coefficients tend to the gaussian function. Plotted are the coefficients of Zt in (.5 + .5Z)20.
Given a sample of a zero mean random time series xt, we may define another series yt by yt = x2t. The problem of estimating the variance of xt is identical to the problem of estimating the mean m of yt. If the sample is short, we may expect an error in our estimate of the variance. Thus, in a scientific paper one would like to write for the mean
Of course (35) really is not right because we really should add something to indicate additional uncertainty due to error in . This estimated error would again have an error, ad infinitum. To really express the result properly, it is necessary to have a probability density function to calculate all the E(xn) which are required. The probability function can be either estimated from the data or chosen theoretically. In practice, for a reason given in a later section, the gaussian function often occurs. In the exercise it is shown that
Correlation is a concept similar to cosine. A cosine measures the angle between two vectors. It is given by the dot product of the two vectors divided by their magnitudes
Correlation is the same sort of thing, except x and y are scalar random variables, so instead of having a vector subscript their subscript is the implicit ensemble subscript. Correlation is defined
In practice one never has an ensemble. There is a practical problem when the ensemble average is simulated by averaging over a sample. The problem arises with small samples and is most dramatically illustrated for a sample with only one element. Then the sample correlation is
regardless of what value the random number x or the random number y should take. In fact, it turns out that the sample correlation will always scatter away from zero.
No doubt this accounts for many false ``discoveries''. The topic of bias and variance of coherency estimates is a complicated one, but a rule of thumb seems to be to expect bias and variance of on the order of for samples of size n.
Find the scale factor s so that . Calculate .Define a reasonable . Examine the uncertainty relation.