(19) |

Next we realize that the data might not be adequate to determine the model, perhaps because our comfortable dense sampling of the model ill fits our economical sparse sampling of data. Thus we adopt a fitting goal that mathematicians call ``regularization'' and we might call a ``model style'' goal or more simply, a quantification of our prejudice about models. We express this by choosing an operator ,often simply a roughener like a gradient (the choice again a topic in this and later chapters). It defines our model residual by or , say we choose

(20) |

In an ideal world, our model prejudice would not conflict with measured data, however, life is not so simple. Since conflicts between data and preconceived notions invariably arise (and they are why we go to the expense of acquiring data) we need an adjustable parameter that measures our ``bullheadedness'', how much we intend to stick to our preconceived notions in spite of contradicting data. This parameter is generally called epsilon because we like to imagine that our bullheadedness is small. (In mathematics, is often taken to be an infinitesimally small quantity.) Although any bullheadedness seems like a bad thing, it must be admitted that measurements are imperfect too. Thus as a practical matter we often find ourselves minimizing

(21) |

(22) |

(23) | ||

(24) |

(25) |

Computationally, we could choose a new with each iteration, but it is more expeditious to freeze , solve the problem, recompute , and solve the problem again. I have never seen a case where more than one iteration was necessary.

People who work with small problems
(less than about 10^{3} vector components)
have access to an attractive theoretical approach
called cross-validation.
Simply speaking,
we could solve the problem many times,
each time omitting a different data value.
Each solution would provide a model
that could be used to predict
the omitted data value.
The quality of these predictions
is a function of and this provides a guide to finding it.
My objections to cross validation are two-fold:
First, I don't know how to apply it in the large problems
like we solve in this book
(I should think more about it);
and second,
people who worry much about ,perhaps first should think
more carefully about
their choice of the filters and ,which is the focus of this book.
Notice that both and can be defined with a scaling factor which is like scaling .Often more important in practice,
with and we have a scaling factor that need not be constant but
can be a function of space or spatial frequency
within the data space and/or model space.

4/27/2004