Operator approach to missing data

missing data

For the operator approach to the fitting goal $-\bold F_k \bold m_k \approx \bold F_u \bold m_u$we rewrite it as $-\bold F_k \bold m_k \approx \bold F \bold J \bold m$ where  

$$-\bold F_k \bold m_k \quad \approx \quad \left[ \begin{arra... ... m_6 \end{array} \right] \quad =\quad \bold F \bold J \bold m$$ (3)

Notice the introduction of the new diagonal matrix $\bold J$,called a masking matrix or a constraint-mask matrix because it multiplies constrained variables by zero leaving freely adjustable variables untouched. Experience shows that a better name than ``mask matrix'' is ``selector matrix'' because what comes out of it, that which is selected, is a less-confusing name for it than which is rejected. With a selector matrix the whole data space seems freely adjustable, both the missing data values and known values. We see that the CD method does not change the known (constrained) values. In general, we derive the fitting goal (3) by

$$\begin{eqnarray} \bold 0 &\approx& \bold F \bold m \\ \bold 0 &\approx& \bold ... ...0 \quad\approx\quad \bold r &=& \bold F\bold J\bold m + \bold r_0\end{eqnarray}$$ (4) (5) (6) (7) (8)

As usual, we find a direction to go $\Delta \bold m$by the gradient of the residual energy.  

$$\Delta\bold m \eq {\partial \over\partial \bold m'}\ \bold r... ... F' + \bold r'_0) \right) \bold r \eq \bold J' \bold F' \bold r$$ (9)

We begin the calculation with the known data values where missing data values are replaced by zeros, namely $(\bold I-\bold J)\bold m$.Filter this data, getting $\bold F(\bold I-\bold J)\bold m$,and load it into the residual $\bold r_0$.With this initialization completed, we begin an iteration loop. First we compute $\Delta m$ from equation (9).

$$\Delta\bold m \quad\longleftarrow\quad \bold J' \bold F' \bold r$$ (10)

$\bold F'$ applies a crosscorrelation of the filter to the residual and then $\bold J'$ sets to zero any changes proposed to known data values. Next, compute the change in residual $\Delta\bold r$from the proposed change in the data $\Delta \bold m$. 

$$\Delta\bold r \quad \longleftarrow \quad \bold F \bold J \Delta \bold m$$ (11)

This applies the filtering again. Then use the method of steepest descent (or conjugate direction) to choose the appropriate scaling (or inclusion of previous step) of $\Delta \bold m$ and $\Delta\bold r$,and update $\bold m$ and $\bold r$ accordingly and iterate.

I could have passed a new operator $\bold F \bold J$into the old solver, but found it worthwhile to write a new, more powerful solver having built-in constraints. To introduce the masking operator $\bold J$ into the solver_smp subroutine , I introduce an optional operator Jop, which is initialized with a logical array of the model size. Two lines in the solver_tiny module

stat = Fop( T, F, g, rd)                          #  g = F' Rd
stat = Fop( F, F, g, gd)                          #  G = F  g

become three lines in the standard library module solver_smp. (We use a temporary array tm of the size of model space.) $\Delta \bold m$ is g and $\Delta\bold r$ is gg.

stat = Fop( T, F, g, rd)                                  # g = F' Rd
if ( present( Jop)) { tm=g;  stat= Jop( F, F, tm, g)      # g = J g
stat = Fop( F, F, g, gg)                                  # G = F g

The full code includes all the definitions we had earlier in solver_tiny module . Merging it with the above bits of code we have the simple solver solver_smp. solver_smpsimple solver

There are two methods of invoking the solver. Comment cards in the code indicate the slightly more verbose method of solution which matches the theory presented in the book.

The subroutine to find missing data is mis1(). It assumes that zero values in the input data correspond to missing data locations. It uses our convolution operator tcai1() . You can also check the Index for other operators and modules. mis11-D missing data

I sought reference material on conjugate gradients with constraints and didn't find anything, leaving me to fear that this chapter was in error and that I had lost the magic property of convergence in a finite number of iterations. I tested the code and it did converge in a finite number of iterations. The explanation is that these constraints are almost trivial. We pretended we had extra variables, and computed a $\Delta \bold m =\bold g$ for each of them. Then we set the $\Delta \bold m =\bold g$ to zero, hence making no changes to anything, like as if we had never calculated the extra $\Delta \bold m$'s.