next up previous [pdf]

Next: GOALS Up: REGULARIZATION Previous: REGULARIZATION

Code modifications required by regularization

Consider regularization of the form $ 0\approx u_\tau - u_{-\tau}$ . In matrix form this is $ \mathbf{0} \approx \mathbf{r}_m = \mathbf{J} \mathbf{u}$ where the matrix $ \mathbf{J}$ is defined below with six vector components in the ordering required by the fast Fourier transform program.
\begin{displaymath}\mathbf{0} \ \approx\
\left[
\begin{array}{c}
r_m(1)\\
r_m(...
...u(1)\\
u(2)\\
u(3)\\
u(4)\\
u(5)\\
u(6)
\end{array}\right]\end{displaymath}     (30)

Note that $ \mathbf{J}^\ast =\mathbf{J}$ . The gradient search direction is
$\displaystyle \Delta\mathbf{u} \ =\
\frac
{\partial\,\mathbf{r}_m^\ast \mathbf...
...\,
{\mathbf{W} \mathbf{r}_m}
\ \ = \ \
\mathbf{J}^\ast \mathbf{W} \mathbf{r}_m$     (31)

where $ \mathbf{W}$ is a diagonal matrix of weights. Again for six components, the diagonal contains $ (1,w_1,w_2,0,w_2,w_1)$ .

Here are the modifications needed to incorporate $ \ell_2$ regularization on $ u_\tau$ :

$\displaystyle {\rm argmin}_{\bold u}\ $   $\displaystyle \sum_{t,x} H(q_t) + \frac{\epsilon}{2} \,
\mathbf{u}^\ast \mathbf{J}^\ast \mathbf{W} \mathbf{J} \mathbf{u}$ (32)
$\displaystyle \Delta u_t$ $\displaystyle =$ $\displaystyle {\rm as\ before }\ + \epsilon \, \mathbf{J}^\ast \mathbf{W} \mathbf{r}_m$ (33)
$\displaystyle \Delta r_t$ $\displaystyle =$ $\displaystyle {\rm as\ before}$ (34)
$\displaystyle \alpha$ $\displaystyle =$ $\displaystyle -\ \frac{(\sum_{x,t} \Delta q_t H_t' ) \
+\ \epsilon\,\ (\bold r...
...elta q_t)^2 H_t'' ) \
+\ \epsilon\,(\Delta \bold r_m \cdot \Delta \bold r_m) }$ (35)

In a least squares problem we compute a step size $ \alpha$ as minus a ratio $ \bold r \cdot \Delta \bold r$ over $ \Delta \bold r \cdot \Delta \bold r$ . Adding a least squares regularization to any convex fitting problem we simply add $ \epsilon\, (\bold r_m \cdot \Delta \bold r_m)$ to the numerator and $ \epsilon \,(\Delta \bold r_m \cdot \Delta \bold r_m)$ to the denominator.

Actually, another regularization is desireable. We should also request $ u_\tau$ to be small for large anticausal lags, lags more negative than the range we are considering for the antisymmetry regularization. This might be handled by truncating the gradient rather than as a regularization.

A third regularization can be added to weaken $ u_\tau$ at its large positive lags in circumstances where we feel we have insufficient data to estimate trace-long filters.


next up previous [pdf]

Next: GOALS Up: REGULARIZATION Previous: REGULARIZATION

2012-05-10