Exercise 4.27: Matrix fractional minimization using second-order cone programming

% From Boyd & Vandenberghe, "Convex Optimization"
% Joëlle Skaf - 09/26/05
%
% Shows the equivalence of the following formulations:
% 1)        minimize    (Ax + b)'*inv(I + B*diag(x)*B')*(Ax + b)
%               s.t.    x >= 0
% 2)        minimize    (Ax + b)'*inv(I + B*Y*B')*(Ax + b)
%               s.t.    x >= 0
%                       Y = diag(x)
% 3)        minimize    v'*v + w'*inv(diag(x))*w
%               s.t.    v + Bw = Ax + b
%                       x >= 0
% 4)        minimize    v'*v + w'*inv(Y)*w
%               s.t.    Y = diag(x)
%                       v + Bw = Ax + b
%                       x >= 0

% Generate input data
randn('state',0);
m = 16; n = 8;
A = randn(m,n);
b = randn(m,1);
B = randn(m,n);

% Problem 1: original formulation
disp('Computing optimal solution for 1st formulation...');
cvx_begin
    variable x1(n)
    minimize( matrix_frac(A*x1 + b , eye(m) + B*diag(x1)*B') )
    x1 >= 0;
cvx_end
opt1 = cvx_optval;

% Problem 2: original formulation (modified)
disp('Computing optimal solution for 2nd formulation...');
cvx_begin
    variable x2(n)
    variable Y(n,n) diagonal
    minimize( matrix_frac(A*x2 + b , eye(m) + B*Y*B') )
    x2 >= 0;
    Y == diag(x2);
cvx_end
opt2 = cvx_optval;

% Problem 3: equivalent formulation (as given in the book)
disp('Computing optimal solution for 3rd formulation...');
cvx_begin
    variables x3(n) w(n) v(m)
    minimize( square_pos(norm(v)) + matrix_frac(w, diag(x3)) )
    v + B*w == A*x3 + b;
    x3 >= 0;
cvx_end
opt3 = cvx_optval;

% Problem 4: equivalent formulation (modified)
disp('Computing optimal solution for 4th formulation...');
cvx_begin
    variables x4(n) w(n) v(m)
    variable Y(n,n) diagonal
    minimize( square_pos(norm(v)) + matrix_frac(w, Y) )
    v + B*w == A*x4 + b;
    x4 >= 0;
    Y == diag(x4);
cvx_end
opt4 = cvx_optval;

% Display the results
disp('------------------------------------------------------------------------');
disp('The optimal value for each of the 4 formulations is: ');
[opt1 opt2 opt3 opt4]
disp('They should be equal!')
Computing optimal solution for 1st formulation...
 
Calling SeDuMi: 161 variables (0 free), 152 equality constraints
------------------------------------------------------------------------
SeDuMi 1.1 by AdvOL, 2005 and Jos F. Sturm, 1998, 2001-2003.
Alg = 2: xz-corrector, Adaptive Step-Differentiation, theta = 0.250, beta = 0.500
eqs m = 152, order n = 26, dim = 298, blocks = 2
nnz(A) = 1368 + 0, nnz(ADA) = 23104, nnz(L) = 11628
 it :     b*y       gap    delta  rate   t/tP*  t/tD*   feas cg cg  prec
  0 :            1.15E+001 0.000
  1 :  2.36E+000 3.77E+000 0.000 0.3268 0.9000 0.9000   0.59  1  1  1.1E+001
  2 :  3.97E+000 1.25E+000 0.000 0.3321 0.9000 0.9000   0.99  1  1  3.5E+000
  3 :  4.75E+000 2.95E-001 0.000 0.2355 0.9000 0.9000   1.14  1  1  7.4E-001
  4 :  5.06E+000 8.69E-002 0.000 0.2948 0.9000 0.9000   1.00  1  1  2.2E-001
  5 :  5.16E+000 1.99E-002 0.000 0.2286 0.9000 0.9000   1.03  1  1  5.0E-002
  6 :  5.18E+000 1.38E-003 0.463 0.0693 0.9902 0.9900   1.02  1  1  3.3E-003
  7 :  5.18E+000 2.33E-004 0.000 0.1692 0.9070 0.9000   1.00  1  1  5.0E-004
  8 :  5.18E+000 2.18E-005 0.000 0.0937 0.9450 0.9450   1.00  1  1  4.7E-005
  9 :  5.18E+000 9.85E-007 0.091 0.0451 0.9900 0.9903   1.00  1  1  2.5E-006
 10 :  5.18E+000 1.35E-007 0.000 0.1371 0.9106 0.9000   1.00  2  1  3.0E-007
 11 :  5.18E+000 9.81E-009 0.486 0.0727 0.9900 0.9903   1.00  2  2  2.4E-008
 12 :  5.18E+000 1.66E-009 0.000 0.1691 0.9000 0.9068   1.00  2  2  4.5E-009

iter seconds digits       c*x               b*y
 12      0.4   Inf  5.1824774137e+000  5.1824774140e+000
|Ax-b| =  1.5e-008, [Ay-c]_+ =  1.7E-010, |x|= 1.4e+001, |y|= 7.2e+000

Detailed timing (sec)
   Pre          IPM          Post
0.000E+000    3.505E-001    1.001E-002    
Max-norms: ||b||=1.488490e+000, ||c|| = 1,
Cholesky |add|=0, |skip| = 0, ||L.L|| = 2.64204.
------------------------------------------------------------------------
Status: Solved
Optimal value (cvx_optval): +5.18248
Computing optimal solution for 2nd formulation...
 
Calling SeDuMi: 161 variables (0 free), 152 equality constraints
------------------------------------------------------------------------
SeDuMi 1.1 by AdvOL, 2005 and Jos F. Sturm, 1998, 2001-2003.
Alg = 2: xz-corrector, Adaptive Step-Differentiation, theta = 0.250, beta = 0.500
eqs m = 152, order n = 26, dim = 298, blocks = 2
nnz(A) = 1368 + 0, nnz(ADA) = 23104, nnz(L) = 11628
 it :     b*y       gap    delta  rate   t/tP*  t/tD*   feas cg cg  prec
  0 :            1.15E+001 0.000
  1 :  2.36E+000 3.77E+000 0.000 0.3268 0.9000 0.9000   0.59  1  1  1.1E+001
  2 :  3.97E+000 1.25E+000 0.000 0.3321 0.9000 0.9000   0.99  1  1  3.5E+000
  3 :  4.75E+000 2.95E-001 0.000 0.2355 0.9000 0.9000   1.14  1  1  7.4E-001
  4 :  5.06E+000 8.69E-002 0.000 0.2948 0.9000 0.9000   1.00  1  1  2.2E-001
  5 :  5.16E+000 1.99E-002 0.000 0.2286 0.9000 0.9000   1.03  1  1  5.0E-002
  6 :  5.18E+000 1.38E-003 0.463 0.0693 0.9902 0.9900   1.02  1  1  3.3E-003
  7 :  5.18E+000 2.33E-004 0.000 0.1692 0.9070 0.9000   1.00  1  1  5.0E-004
  8 :  5.18E+000 2.18E-005 0.000 0.0937 0.9450 0.9450   1.00  1  1  4.7E-005
  9 :  5.18E+000 9.85E-007 0.091 0.0451 0.9900 0.9903   1.00  1  1  2.5E-006
 10 :  5.18E+000 1.35E-007 0.000 0.1371 0.9106 0.9000   1.00  2  1  3.0E-007
 11 :  5.18E+000 9.81E-009 0.486 0.0727 0.9900 0.9903   1.00  2  2  2.4E-008
 12 :  5.18E+000 1.66E-009 0.000 0.1691 0.9000 0.9068   1.00  2  2  4.5E-009

iter seconds digits       c*x               b*y
 12      0.3   Inf  5.1824774137e+000  5.1824774140e+000
|Ax-b| =  1.5e-008, [Ay-c]_+ =  1.7E-010, |x|= 1.4e+001, |y|= 7.2e+000

Detailed timing (sec)
   Pre          IPM          Post
4.006E-002    3.405E-001    0.000E+000    
Max-norms: ||b||=1.488490e+000, ||c|| = 1,
Cholesky |add|=0, |skip| = 0, ||L.L|| = 2.64204.
------------------------------------------------------------------------
Status: Solved
Optimal value (cvx_optval): +5.18248
Computing optimal solution for 3rd formulation...
 
Calling SeDuMi: 73 variables (0 free), 54 equality constraints
------------------------------------------------------------------------
SeDuMi 1.1 by AdvOL, 2005 and Jos F. Sturm, 1998, 2001-2003.
Alg = 2: xz-corrector, Adaptive Step-Differentiation, theta = 0.250, beta = 0.500
eqs m = 54, order n = 22, dim = 111, blocks = 4
nnz(A) = 319 + 0, nnz(ADA) = 2740, nnz(L) = 1397
 it :     b*y       gap    delta  rate   t/tP*  t/tD*   feas cg cg  prec
  0 :            4.49E+000 0.000
  1 :  1.71E+000 2.20E+000 0.000 0.4902 0.9000 0.9000   0.91  1  1  7.1E+000
  2 :  2.79E+000 6.56E-001 0.000 0.2981 0.9000 0.9000   1.46  1  1  1.7E+000
  3 :  4.27E+000 1.77E-001 0.000 0.2699 0.9000 0.9000   1.13  1  1  4.3E-001
  4 :  4.91E+000 4.32E-002 0.000 0.2438 0.9000 0.9000   1.03  1  1  1.0E-001
  5 :  5.13E+000 7.87E-003 0.000 0.1822 0.9000 0.9000   0.99  1  1  1.9E-002
  6 :  5.17E+000 1.42E-003 0.000 0.1810 0.9000 0.9005   0.99  1  1  3.6E-003
  7 :  5.18E+000 1.27E-004 0.000 0.0889 0.8199 0.9000   1.00  1  1  8.4E-004
  8 :  5.18E+000 6.97E-006 0.000 0.0551 0.9900 0.9900   1.00  1  1  4.6E-005
  9 :  5.18E+000 1.25E-006 0.000 0.1799 0.9000 0.9000   1.00  1  1  8.4E-006
 10 :  5.18E+000 1.55E-007 0.224 0.1239 0.9450 0.9450   1.00  1  1  1.0E-006
 11 :  5.18E+000 2.99E-008 0.064 0.1924 0.9000 0.9000   1.00  1  1  2.0E-007
 12 :  5.18E+000 5.67E-009 0.148 0.1899 0.9000 0.9000   1.00  2  2  3.8E-008
 13 :  5.18E+000 1.06E-009 0.155 0.1860 0.9000 0.9000   1.00  3  3  7.0E-009

iter seconds digits       c*x               b*y
 13      0.2   Inf  5.1824773967e+000  5.1824773987e+000
|Ax-b| =  1.2e-008, [Ay-c]_+ =  7.6E-010, |x|= 6.2e+000, |y|= 1.1e+001

Detailed timing (sec)
   Pre          IPM          Post
0.000E+000    1.803E-001    0.000E+000    
Max-norms: ||b||=1.488490e+000, ||c|| = 1,
Cholesky |add|=1, |skip| = 0, ||L.L|| = 500000.
------------------------------------------------------------------------
Status: Solved
Optimal value (cvx_optval): +5.18248
Computing optimal solution for 4th formulation...
 
Calling SeDuMi: 73 variables (0 free), 54 equality constraints
------------------------------------------------------------------------
SeDuMi 1.1 by AdvOL, 2005 and Jos F. Sturm, 1998, 2001-2003.
Alg = 2: xz-corrector, Adaptive Step-Differentiation, theta = 0.250, beta = 0.500
eqs m = 54, order n = 22, dim = 111, blocks = 4
nnz(A) = 319 + 0, nnz(ADA) = 2740, nnz(L) = 1397
 it :     b*y       gap    delta  rate   t/tP*  t/tD*   feas cg cg  prec
  0 :            4.49E+000 0.000
  1 :  1.71E+000 2.20E+000 0.000 0.4902 0.9000 0.9000   0.91  1  1  7.1E+000
  2 :  2.79E+000 6.56E-001 0.000 0.2981 0.9000 0.9000   1.46  1  1  1.7E+000
  3 :  4.27E+000 1.77E-001 0.000 0.2699 0.9000 0.9000   1.13  1  1  4.3E-001
  4 :  4.91E+000 4.32E-002 0.000 0.2438 0.9000 0.9000   1.03  1  1  1.0E-001
  5 :  5.13E+000 7.87E-003 0.000 0.1822 0.9000 0.9000   0.99  1  1  1.9E-002
  6 :  5.17E+000 1.42E-003 0.000 0.1810 0.9000 0.9005   0.99  1  1  3.6E-003
  7 :  5.18E+000 1.27E-004 0.000 0.0889 0.8199 0.9000   1.00  1  1  8.4E-004
  8 :  5.18E+000 6.97E-006 0.000 0.0551 0.9900 0.9748   1.00  1  1  4.6E-005
  9 :  5.18E+000 1.25E-006 0.000 0.1799 0.9000 0.9000   1.00  1  1  8.4E-006
 10 :  5.18E+000 1.55E-007 0.224 0.1239 0.9450 0.9450   1.00  1  1  1.0E-006
 11 :  5.18E+000 3.01E-008 0.064 0.1935 0.9000 0.9000   1.00  1  1  2.0E-007
 12 :  5.18E+000 5.66E-009 0.148 0.1883 0.9000 0.9000   1.00  1  1  3.8E-008
 13 :  5.18E+000 1.04E-009 0.155 0.1834 0.9000 0.9000   1.00  3  3  7.0E-009

iter seconds digits       c*x               b*y
 13      0.1   Inf  5.1824773967e+000  5.1824773987e+000
|Ax-b| =  1.2e-008, [Ay-c]_+ =  7.6E-010, |x|= 6.2e+000, |y|= 1.2e+001

Detailed timing (sec)
   Pre          IPM          Post
0.000E+000    9.013E-002    0.000E+000    
Max-norms: ||b||=1.488490e+000, ||c|| = 1,
Cholesky |add|=1, |skip| = 0, ||L.L|| = 500000.
------------------------------------------------------------------------
Status: Solved
Optimal value (cvx_optval): +5.18248
------------------------------------------------------------------------
The optimal value for each of the 4 formulations is: 

ans =

    5.1825    5.1825    5.1825    5.1825

They should be equal!