next up previous [pdf]

Next: APPENDIX B Up: Berryman: MESA Previous: acknowledgments

APPENDIX A

The following derivation of the relationship between entropy and power spectrum is essentially the same as that given by (). The derivation is included here for completeness.

The entropy of $ N$ random variables $ X_1$, $ \ldots$, $ X_N$ is given by

$\displaystyle H_N = - \int P(X_1,\ldots,X_N) \ln[a^{2N}P(X_1,\ldots,X_N)]d^NX = -2N\ln a - \int P\ln Pd^NX,$ (A-1)

where $ P$ is the joint probability density and $ a$ is a constant with the same units as $ X$. The power spectrum $ P(f)$ computed from the autocorrelation values $ R_0, \ldots, R_{N-1}$ depends only on the second-order statistics of the time series $ \left\{X_n\right\}$. Therefore, the given time series cannot be distinguished from a normal (Gaussian) process.

The joint probability density for a normal process with $ N$ variables of zero mean is (using matrix notation, where $ X^T$ is the transpose of $ X$)

$\displaystyle P(X_1,\ldots,X_N) = \left[\left(2\pi e\right)^N\det T_{N-1}\right]^{-\frac{1}{2}} \exp\left(-\frac{1}{2}X^T\cdot T^{-1}_{N-1}\cdot X\right),$ (A-2)

where $ T_{N-1}$ is the $ N\times N$ Toeplitz matrix [see ()] given by Equation (11) and $ X$ is the $ N$-vector determined by $ X^T = \left(X_1,\ldots,X_N\right)$. Substituting (A-2) into (A-1), we find

$\displaystyle H_N = \frac{1}{2}\ln\left[(2\pi e)^N\det T_{N-1}\right]-2N\ln a.$ (A-3)

Setting the arbitrary constant $ a = \left(2\pi e\right)^{1/4}$ for convenience, Equation (A-3) then becomes

$\displaystyle H_N = \frac{1}{2}\ln\left(\det T_{N-1}\right).$ (A-4)

Since (A-4) necessarily diverges as $ N\to \infty$, a better measure of the information content of the series is the average entropy per variable given by

$\displaystyle h = \lim_{N\to \infty} \frac{H_N}{N} = \lim_{N\to \infty} \ln\left(\det T_{N-1}\right)^{1/2N}.$ (A-5)

The eigenvalues $ \left\{\lambda_1, \ldots, \lambda_N\right\}$ of $ T_{N-1}$ are real and nonnegative since $ T$ is Hermitian and nonnegative definite. Furthermore,

$\displaystyle \det T_{N-1} = \Pi_{i=1}^N \lambda_i,$ (A-6)

so

$\displaystyle h = \lim_{N\to\infty} \frac{1}{2N} \sum_{i=1}^N \ln \lambda_i.$ (A-7)

The Szëgo theorem (, ,) states that, if $ F$ is any continuous function, then

$\displaystyle \lim_{N\to \infty} \frac{1}{N}\sum_{i=1}^N F(\lambda_i) = \frac{1}{2W}\int_{-W}^W F[2WP(f)] \,df,$ (A-8)

where, as before, $ W$ is the Nyquist frequency, $ P(f)$ is the power spectrum, and the $ \lambda$'s are the $ N$ eigenvalues of $ T_{N-1}$.

Combining Equations (A-7) and (A-8), we find

$\displaystyle h = \frac{1}{4W}\int^W_{-W} \ln[2WP(f)]\,df,$ (A-9)

which is the sought after result.


next up previous [pdf]

Next: APPENDIX B Up: Berryman: MESA Previous: acknowledgments

2009-04-13