next up previous print clean
Next: RELATED CONCEPTS Up: THE JENSEN INEQUALITY Previous: THE JENSEN INEQUALITY

Examples of Jensen inequalities

The most familiar example of a Jensen inequality occurs when the weights are all equal to 1/N and the convex function is f(x) = x2. In this case the Jensen inequality gives the familiar result that the mean square exceeds the square of the mean:
\begin{displaymath}
Q \eq {1\over N}\sum_{i=1}^N x_i^2\ -\ \left( {1\over N}
 \sum_{i=1}^N x_i \right)^2 \quad \geq \quad 0\end{displaymath} (4)
In the other applications we will consider, the population consists of positive members, so the function f(p) need have a positive second derivative only for positive values of p. The function f(p)=1/p yields a Jensen inequality for the harmonic mean:
\begin{displaymath}
H \eq \sum {w_i\over p_i}\ -\ {1\over \sum w_i p_i} \quad \geq \quad 0\end{displaymath} (5)
A more important case is the geometric inequality. Here $f(p) = - \ln (p)$, and
\begin{displaymath}
G \eq -\sum w_i \ln p_i\ +\ \ln \sum w_i p_i \quad \geq \quad 0\end{displaymath} (6)
The more familiar form of the geometric inequality results from exponentiation and a choice of weights equal to 1/N:
\begin{displaymath}
{1\over N}\sum_{i=1}^N p_i \quad \geq \quad \prod_{i=1}^N p_i^{1/N}\end{displaymath} (7)
In other words, the product of square roots of two values is smaller than half the sum of the values.

A Jensen inequality with an adjustable parameter is suggested by $f(p)=p^\gamma$:
\begin{displaymath}
\Gamma_\gamma \eq \sum_{i=1}^N w_i p_i^\gamma\ -\ 
 \left( \sum_{i=1}^N w_i p_i \right)^\gamma\end{displaymath} (8)
Whether $\Gamma$ is always positive or always negative depends upon the numerical value of $\gamma$.In practice we may see the dimensionless form, in which the ratio instead of the difference of the two terms is used.

A most important inequality in information theory and thermodynamics is the one based on $f(p)=p^{1+\epsilon}$,where $\epsilon$ is a small positive number tending to zero. I call this the ``weak" inequality. With some calculation we will quickly arrive at the limit:
\begin{displaymath}
\sum w_i p_i^{1+\epsilon} \quad \geq \quad \left( \sum
 w_i p_i \right)^{1+\epsilon}\end{displaymath} (9)
Take logarithms
\begin{displaymath}
\ln \sum w_i p_i^{1+\epsilon} \quad \geq \quad (1+\epsilon)
 \ln \sum w_i p_i\end{displaymath} (10)
Expand both sides in a Taylor series in powers of $\epsilon$ using
\begin{displaymath}
{d\over d\epsilon} a^u \eq {du\over d\epsilon} a^u \ln a\end{displaymath} (11)
The leading term is identical on both sides and can be canceled. Divide both sides by $\epsilon$ and go to the limit $\epsilon=0$,obtaining
\begin{displaymath}
{\sum w_i p_i \ln p_i\over \sum w_i p_i} \quad \geq \quad 
 \ln \sum w_i p_i\end{displaymath} (12)
We can now define a positive variable S' with or without a positive scaling factor $\sum w p$:
\begin{eqnarray}
S'_{\rm intensive} &=&{\sum w_i p_i \ln p_i\over 
 \sum w_i p_i...
 ... w_i p_i\right)\ \ln \left( \sum w_i p_i\right) \quad \geq \quad 0\end{eqnarray} (13)
(14)
Seismograms often contain zeros and gaps. Notice that a single zero pi can upset the harmonic H or geometric G inequality, but a single zero has no horrible effect on S or $\Gamma$.


next up previous print clean
Next: RELATED CONCEPTS Up: THE JENSEN INEQUALITY Previous: THE JENSEN INEQUALITY
Stanford Exploration Project
10/21/1998