THE JENSEN INEQUALITY

Let f be a function with a positive second derivative. Such a function is called ``convex" and satisfies the inequality  

$${f(a)\ +\ f(b)\over 2}\ -\ f\left( {a + b\over 2}\right)\quad \geq \quad 0$$ (1)

Equation (1) relates a function of an average to an average of the function. The average can be weighted, for example,  

$${ \frac{1}{3} \, f(a)\ +\ \frac{2}{3} \, f(b)}\ -\ f\left( { \frac{1}{3} a + \frac{2}{3} b}\right) \quad \geq \quad 0$$ (2)

Figure 1 is a graphical interpretation of equation (2) for the function f=x².

 

jen Figure 1 Sketch of y=x2 for interpreting equation ((2)).

There is nothing special about f=x², except that it is convex. Given three numbers a, b, and c, the inequality (2) can first be applied to a and b, and then to c and the average of a and b. Thus, recursively, an inequality like (2) can be built for a weighted average of three or more numbers. Define weights $w_j \geq 0$ that are normalized (${\sum_j} w_j = 1$). The general result is  

$$S(p_j) \eq \sum_{j=1}^N w_j f(p_j)\ -\ f\left( \sum_{j=1}^N w_j p_j \right) \quad \geq \quad 0$$ (3)

If all the p_j are the same, then both of the two terms in S are the same, and S vanishes. Hence, minimizing S is like urging all the p_j to be identical. Equilibrium is when S is reduced to the smallest possible value which satisfies any constraints that may be applicable. The function S defined by (3) is like the entropy defined in thermodynamics.