Preconditioning

  When I first realized that practical imaging methods in widespread industrial use amounted merely to the adjoint of forward modeling, I (and others) thought an easy way to achieve fame and fortune would be to introduce the first steps towards inversion along the lines of Chapter . Although inversion generally requires a prohibitive number of steps, I felt that moving in the gradient direction, the direction of steepest descent, would move us rapidly in the direction of practical improvements. This turned out to be optimistic. It was too slow. But then I learned about the conjugate gradient method that spectacularly overcomes a well-known speed problem with the method of steepest descents. I came to realize that it was still too slow. I learned this by watching the convergence in Figure . This led me to the helix method in Chapter . Here we'll see how it speeds many applications.

We'll also come to understand why the gradient is such a poor direction both for steepest descent and for conjugate gradients. An indication of our path is found in the contrast between and exact solution $\bold m = (\bold A'\bold A)^{-1}\bold A'\bold d$ and the gradient $\Delta \bold m = \bold A'\bold d$(which is the first step starting from $\bold m =\bold 0$). Notice that $\Delta \bold m$ differs from $\bold m$by the factor $(\bold A'\bold A)^{-1}$.This factor is sometimes called a spectrum and in some situations it literally is a frequency spectrum. In these cases, $\Delta \bold m$ simply gets a different spectrum from $\bold m$ and many iterations are required to fix it. Here we'll find that for many problems, ``preconditioning'' with the helix is a better way.