Example 7.2: Maximum entropy distribution
n = 100;
a = linspace(-1,1,n);
a2 = a .^ 2;
a3 = 3 * ( a.^ 3 ) - 2 * a;
ap = +( a < 0 );
A = [ a ; -a ; a2 ; -a2 ; a3 ; -a3 ; ap ; -ap ];
b = [ 0.1 ; 0.1 ;0.5 ; -0.5 ; -0.2 ; 0.3 ; 0.4 ; -0.3 ];
cvx_begin
cvx_gp_precision( 0.001 )
variables z(8) y(100)
dual variable pent
minimize( b' * z + logsumexp_sdp( y ) )
pent : y + A' * z == 0;
z >= 0;
cvx_end
cvxq = cvx_quiet(true);
Ubnds = zeros(1,n);
Lbnds = zeros(1,n);
for t = 1 : n,
cvx_begin
variable p( n )
minimize sum( p(1:t) )
p >= 0; sum( p ) == 1;
A * p <= b;
cvx_end
Lbnds(t) = cvx_optval;
cvx_begin
variable p( n )
maximize sum( p(1:t) )
p >= 0; sum( p ) == 1;
A * p <= b;
cvx_end
Ubnds(t) = cvx_optval;
disp( sprintf( '%g <= Prob(x<=%g) <= %g', Lbnds(t), a(t), Ubnds(t) ) );
end
cvx_quiet(cvxq);
figure( 1 )
stairs( a, pent );
xlabel( 'x' );
ylabel( 'PDF( x )' );
figure( 2 )
stairs( a, cumsum( pent ) );
grid on;
hold on
d = stairs(a, Lbnds,'r-'); set(d,'Color',[0 0.5 0]);
d = stairs(a, Ubnds,'r-'); set(d,'Color',[0 0.5 0]);
d = plot([-1,-1], [Lbnds(1), Ubnds(1)],'r-');
set(d,'Color',[0 0.5 0]);
axis([-1.1 1.1 -0.1 1.1]);
xlabel( 'x' );
ylabel( 'CDF( x )' );
hold off
Calling SeDuMi: 1988 variables (99 free), 1089 equality constraints
------------------------------------------------------------------------
SeDuMi 1.1 by AdvOL, 2005 and Jos F. Sturm, 1998, 2001-2003.
Alg = 2: xz-corrector, Adaptive Step-Differentiation, theta = 0.250, beta = 0.500
Split 99 free variables
eqs m = 1089, order n = 1098, dim = 3078, blocks = 100
nnz(A) = 4145 + 600, nnz(ADA) = 11981, nnz(L) = 6810
Handling 6 + 0 dense columns.
it : b*y gap delta rate t/tP* t/tD* feas cg cg prec
0 : 9.31E-002 0.000
1 : 1.06E+001 5.30E-002 0.000 0.5696 0.9000 0.9000 2.58 1 1 3.3E+000
2 : 1.46E+001 1.59E-002 0.000 0.3003 0.9000 0.9000 2.27 1 1 5.9E-001
3 : 1.13E+001 5.28E-003 0.000 0.3314 0.9000 0.9000 1.58 1 1 1.5E-001
4 : 8.26E+000 2.21E-003 0.000 0.4184 0.9000 0.9000 1.39 1 1 5.6E-002
5 : 6.61E+000 1.09E-003 0.000 0.4936 0.9000 0.9000 1.18 1 1 2.7E-002
6 : 5.95E+000 7.16E-004 0.000 0.6571 0.9000 0.9000 1.06 1 1 1.8E-002
7 : 5.38E+000 4.07E-004 0.000 0.5685 0.9000 0.9000 1.05 1 1 1.0E-002
8 : 4.95E+000 2.14E-004 0.000 0.5257 0.9000 0.9000 1.04 1 1 5.3E-003
9 : 4.95E+000 3.71E-006 0.000 0.0173 0.9000 0.0000 1.04 1 1 2.4E-003
10 : 4.61E+000 6.31E-007 0.000 0.1702 0.9219 0.9000 1.00 1 1 5.0E-004
11 : 4.46E+000 1.65E-007 0.000 0.2615 0.9074 0.9000 0.98 1 1 1.3E-004
12 : 4.42E+000 5.66E-008 0.069 0.3429 0.1225 0.9000 0.99 1 1 5.4E-005
13 : 4.40E+000 2.57E-008 0.000 0.4539 0.9000 0.9000 0.97 1 1 2.5E-005
14 : 4.39E+000 5.16E-009 0.000 0.2007 0.9000 0.9000 1.01 1 1 5.0E-006
15 : 4.39E+000 1.01E-009 0.000 0.1949 0.9000 0.9000 1.02 1 1 9.6E-007
16 : 4.39E+000 2.28E-010 0.000 0.2268 0.9000 0.9000 1.03 1 1 2.1E-007
17 : 4.39E+000 5.19E-011 0.000 0.2277 0.9000 0.9000 1.05 1 2 4.7E-008
18 : 4.39E+000 1.07E-011 0.000 0.2068 0.9000 0.9000 1.04 2 2 9.6E-009
iter seconds digits c*x b*y
18 0.6 Inf 4.3882141650e+000 4.3882141729e+000
|Ax-b| = 6.2e-008, [Ay-c]_+ = 7.2E-009, |x|= 6.2e+001, |y|= 2.5e+000
Detailed timing (sec)
Pre IPM Post
0.000E+000 5.908E-001 0.000E+000
Max-norms: ||b||=3.646025e+000, ||c|| = 1.849460e+000,
Cholesky |add|=0, |skip| = 0, ||L.L|| = 18.0857.
------------------------------------------------------------------------
Status: Solved
Optimal value (cvx_optval): +4.38821
0 <= Prob(x<=-1) <= 0.329406
0 <= Prob(x<=-0.979798) <= 0.344777
0 <= Prob(x<=-0.959596) <= 0.360771
0 <= Prob(x<=-0.939394) <= 0.377365
0 <= Prob(x<=-0.919192) <= 0.394532
0 <= Prob(x<=-0.89899) <= 0.4
0 <= Prob(x<=-0.878788) <= 0.4
0 <= Prob(x<=-0.858586) <= 0.4
0 <= Prob(x<=-0.838384) <= 0.4
0 <= Prob(x<=-0.818182) <= 0.4
0 <= Prob(x<=-0.79798) <= 0.4
0.0116304 <= Prob(x<=-0.777778) <= 0.4
0.0331953 <= Prob(x<=-0.757576) <= 0.4
0.0519157 <= Prob(x<=-0.737374) <= 0.4
0.0701091 <= Prob(x<=-0.717172) <= 0.4
0.0859495 <= Prob(x<=-0.69697) <= 0.4
0.0998612 <= Prob(x<=-0.676768) <= 0.4
0.112141 <= Prob(x<=-0.656566) <= 0.4
0.123045 <= Prob(x<=-0.636364) <= 0.4
0.132778 <= Prob(x<=-0.616162) <= 0.4
0.141527 <= Prob(x<=-0.59596) <= 0.4
0.149418 <= Prob(x<=-0.575758) <= 0.4
0.15655 <= Prob(x<=-0.555556) <= 0.4
0.163015 <= Prob(x<=-0.535354) <= 0.4
0.168895 <= Prob(x<=-0.515152) <= 0.4
0.174283 <= Prob(x<=-0.494949) <= 0.4
0.179205 <= Prob(x<=-0.474747) <= 0.4
0.18371 <= Prob(x<=-0.454545) <= 0.4
0.187841 <= Prob(x<=-0.434343) <= 0.4
0.191651 <= Prob(x<=-0.414141) <= 0.4
0.195164 <= Prob(x<=-0.393939) <= 0.4
0.198396 <= Prob(x<=-0.373737) <= 0.4
0.201373 <= Prob(x<=-0.353535) <= 0.4
0.204127 <= Prob(x<=-0.333333) <= 0.4
0.206681 <= Prob(x<=-0.313131) <= 0.4
0.209037 <= Prob(x<=-0.292929) <= 0.4
0.211209 <= Prob(x<=-0.272727) <= 0.4
0.213219 <= Prob(x<=-0.252525) <= 0.4
0.215088 <= Prob(x<=-0.232323) <= 0.4
0.216811 <= Prob(x<=-0.212121) <= 0.4
0.218398 <= Prob(x<=-0.191919) <= 0.4
0.219862 <= Prob(x<=-0.171717) <= 0.4
0.221224 <= Prob(x<=-0.151515) <= 0.4
0.222474 <= Prob(x<=-0.131313) <= 0.4
0.223619 <= Prob(x<=-0.111111) <= 0.4
0.224669 <= Prob(x<=-0.0909091) <= 0.4
0.225643 <= Prob(x<=-0.0707071) <= 0.4
0.22653 <= Prob(x<=-0.0505051) <= 0.4
0.227334 <= Prob(x<=-0.030303) <= 0.4
0.3 <= Prob(x<=-0.010101) <= 0.4
0.3 <= Prob(x<=0.010101) <= 0.778942
0.3 <= Prob(x<=0.030303) <= 0.792532
0.3 <= Prob(x<=0.0505051) <= 0.806483
0.3 <= Prob(x<=0.0707071) <= 0.819022
0.3 <= Prob(x<=0.0909091) <= 0.825
0.3 <= Prob(x<=0.111111) <= 0.83125
0.3 <= Prob(x<=0.131313) <= 0.837791
0.3 <= Prob(x<=0.151515) <= 0.841937
0.3 <= Prob(x<=0.171717) <= 0.845957
0.3 <= Prob(x<=0.191919) <= 0.850137
0.3 <= Prob(x<=0.212121) <= 0.854492
0.3 <= Prob(x<=0.232323) <= 0.859052
0.3 <= Prob(x<=0.252525) <= 0.863811
0.3 <= Prob(x<=0.272727) <= 0.868817
0.3 <= Prob(x<=0.292929) <= 0.874066
0.3 <= Prob(x<=0.313131) <= 0.877055
0.3 <= Prob(x<=0.333333) <= 0.880067
0.3 <= Prob(x<=0.353535) <= 0.883272
0.300787 <= Prob(x<=0.373737) <= 0.886687
0.307695 <= Prob(x<=0.393939) <= 0.890333
0.314397 <= Prob(x<=0.414141) <= 0.894234
0.320909 <= Prob(x<=0.434343) <= 0.898418
0.327232 <= Prob(x<=0.454545) <= 0.902981
0.333379 <= Prob(x<=0.474747) <= 0.909013
0.339323 <= Prob(x<=0.494949) <= 0.916606
0.345134 <= Prob(x<=0.515152) <= 0.925292
0.350719 <= Prob(x<=0.535354) <= 0.935184
0.356201 <= Prob(x<=0.555556) <= 0.946304
0.361491 <= Prob(x<=0.575758) <= 0.958921
0.366603 <= Prob(x<=0.59596) <= 0.973265
0.371622 <= Prob(x<=0.616162) <= 0.989508
0.387329 <= Prob(x<=0.636364) <= 1
0.410495 <= Prob(x<=0.656566) <= 1
0.439031 <= Prob(x<=0.676768) <= 1
0.466372 <= Prob(x<=0.69697) <= 1
0.492663 <= Prob(x<=0.717172) <= 1
0.518025 <= Prob(x<=0.737374) <= 1
0.542592 <= Prob(x<=0.757576) <= 1
0.56651 <= Prob(x<=0.777778) <= 1
0.589941 <= Prob(x<=0.79798) <= 1
0.613125 <= Prob(x<=0.818182) <= 1
0.635881 <= Prob(x<=0.838384) <= 1
0.657609 <= Prob(x<=0.858586) <= 1
0.678314 <= Prob(x<=0.878788) <= 1
0.697846 <= Prob(x<=0.89899) <= 1
0.716238 <= Prob(x<=0.919192) <= 1
0.733536 <= Prob(x<=0.939394) <= 1
0.74974 <= Prob(x<=0.959596) <= 1
0.764914 <= Prob(x<=0.979798) <= 1
1 <= Prob(x<=1) <= 1