Conjugate directions

To construct a set of directions ${\bf p}_n$ which satisfy the conjugacy criterion (preconjugacy), we can start from an arbitrary set of model-space vectors ${\bf c}_n$ and apply an orthogonalization process to their projections in the data space. An iterative orthogonalization is defined by recursion

p_n = c_n - _j=1^n-1_n^(j)p_j,   where the following choice of the scalar coefficients $\beta_n^{(j)}$ assures condition (preconjugacy):

_n^(j) = (M c_n,Mp_j) ||Mp_j||^2.   According to the fact that the residual vector ${\bf r}_n$ is orthogonal to all the previous steps in the data space (equation (preresmpj)), the coefficient $\alpha_n$ simplifies to

_n = (r_n-1,Mc_n-1) ||Mp_n-1||^2.   Formulas (ortoprocess-cdalpha) define the method of conjugate directions Fomel (1996) also known as the preconditioned Krylov subspace method Kleinman and van den Berg (1991) and under several other names.

A particular choice of the initial directions ${\bf c}_n = {\bf g}_n = {\bf M}^T{\bf r}_n$transforms the method of conjugate directions into the method of conjugate gradients and introduces remarkable simplifications.