We start again with the basic equation (1) and
introduce a residual vector ** r**, defining it by the relationship

(10) |

(11) |

(12) |

System (11) is clearly under-determined with respect to the compound model . If from all possible solutions of this system we seek the one with the minimal power , the formal (ideal) result takes the well-known form

(13) |

(14) |

Let us show that estimate (13) is exactly equivalent to estimate (9) from the ``trivial'' model-space regularization. Consider the operator

(15) |

(16) |

(17) |

Not only the optimization estimate, but also the form of the objective function, is exactly equivalent for both data-space and mode-space cases. The objective function of model-space least squares is connected with the data-space objective function by the simple proportionality

(18) |

To move to a more general (and interesting) case of ``non-trivial''
data-space regularization, we need to refer to the concept of model
*preconditioning* Nichols (1994). A preconditioning
operator ** P** is used to introduce a new model

(19) |

(20) |

(21) |

(22) |

(23) |

Now we can show that estimate (23) is exactly equivalent to formula (7) from the model-space regularization under the condition

(24) |

(25) |

(26) |

(27) |

Comparing formulas (23) and (7), it is
interesting to note that we can turn a trivial regularization into a
non-trivial one by simply replacing the exact adjoint operator ** L^{T}**
by the operator

(28) |

Though the final results of the model-space and data-space
regularization are identical, the effect of preconditioning may alter
the behavior of iterative gradient-based methods, such as the method
of conjugate gradients. Though the objective functions are equal,
their gradients with respect to the model parameters are different.
Note, for example, that the first iteration of the model-space
regularization yields ** L^{T} d** as the model estimate regardless of the
regularization operator, while the first iteration of the model-space
regularization yields

The two objective functions produce different results when optimization is incomplete. A descent optimization of the original (model-space -Examples in the next section illustrate these conclusions.S.F.) objective function will begin with complex perturbations of the model and slowly converge toward an increasingly simple model at the global minimum. A descent optimization of the revised (data-space -S.F.) objective function will begin with simple perturbations of the model and slowly converge toward an increasingly complex model at the global minimum. ...A more economical implementation can use fewer iterations. Insufficient iterations result in an insufficiently complex model, not in an insufficiently simplified model.

11/11/1997