Next: Examples of Jensen inequalities Up: Entropy and Jensen inequality Previous: Entropy and Jensen inequality

# THE JENSEN INEQUALITY

Let f be a function with a positive second derivative. Such a function is called convex" and satisfies the inequality
 (1)
Equation (1) relates a function of an average to an average of the function. The average can be weighted, for example,
 (2)
Figure 1 is a graphical interpretation of equation (2) for the function f=x2.

 jen Figure 1 Sketch of y=x2 for interpreting equation ((2)).

There is nothing special about f=x2, except that it is convex. Given three numbers a, b, and c, the inequality (2) can first be applied to a and b, and then to c and the average of a and b. Thus, recursively, an inequality like (2) can be built for a weighted average of three or more numbers. Define weights that are normalized (). The general result is
 (3)
If all the pj are the same, then both of the two terms in S are the same, and S vanishes. Hence, minimizing S is like urging all the pj to be identical. Equilibrium is when S is reduced to the smallest possible value which satisfies any constraints that may be applicable. The function S defined by (3) is like the entropy defined in thermodynamics.

Next: Examples of Jensen inequalities Up: Entropy and Jensen inequality Previous: Entropy and Jensen inequality
Stanford Exploration Project
10/21/1998