Iterative methods of linear unconstrained optimization, such as the
method of conjugate gradients Hestenes and Stiefel (1952),
Lanczos 1950,
LSQR Paige and Saunders (1982),
GMRES Saad and Schultz (1986), and some others, have proven to
be important tools in geophysical applications. For a given forward
modeling operator, predicting the existing data from an unknown model,
iterative methods approach the model, which minimizes the squared
residual error of prediction. In linear problems, the global minimum
does exist. However, finding it requires, in general, the number of
iterative steps equal to the number of unknown model parameters. In
large-scale problems, typical in geophysical applications, the
computational cost makes complete solution practically
infeasible. Nevertheless, iterative methods allow us to get a
reasonable *estimate* of the solution in a reasonable number of
iterations.

When *solving* the inverse problem is replaced by *estimating*
the solution, the inversion theory needs to be reformulated. Methods
and formulations, designed for the complete solutions, are no longer
applicable in the case of iterative estimates. This conclusion applies
to such objects as pseudoinverse operator, model resolution, and data
resolution, conventionally associated with SVD decomposition,
which becomes infeasible in many large-scale problems.

In this paper, we review different methods of iterative optimization, primarily the method of conjugate directions and the LSQR method. We prove that these methods have a common origin in the general principle of the iterative residual minimization. The general principle leads to remarkable orthogonalization properties for particular sets of vectors in the model and data subspaces. Whenever possible, the orthogonality conditions should be enforced in numerical implementations as a warranty of stable iterative behavior.

We show how to define the effective pseudoinverse operator, model and data resolution for iterative methods. Since the exact solution is not available, those definitions apply to effective iterative estimates of the corresponding operators, which were strictly defined in the inversion theory.

Finally, we illustrate the theory with simple examples from crosswell traveltime tomography.

11/11/1997