Open book quiz.   5 minutes.          Your name _____________________________

                                                

Recall Jon's lecture about how his 1973 L1 code worked
maintaining a set of basis equations
orthogonalized by the Gramm-Schmitt method.

Jon realized later he had skimmed over a critical part.
His lecture showed how to maintain the organized set of basis equations
but it presumed "someone" had passed him new basis equations.
How does "someone" find a good equation to add to the basis?


1.   Start from   dx = F' sgn(r)

2.   Remove from dx its projections on the present basis
	(or whatever subset of the present basis you want to keep)

3.   Do line search for alpha minimizing SUM | F(x+alpha dx) - d |
	with a weighted median program.

4.   The equation at the minimum is the new basis equation.


-----------------------------------------------

Most of the horses (gradients) have an adjustable parameter r0
demarking the transition from L2 to L1 behavior.
Without giving any reason, Jon said for a blocky model
using the regularization |dm/dt|
he would set r0 to be the 95th percentile of |r_i|
if he wanted blocks about 20 points long.
He did not give a very deep reason.
Give a good convincing reason relating to basis equations.


ANSWER:   Basis equations are fit exactly.   The trouble with |dm/dt|
is that so many equations need be in the basis.  We can't afford to try.
Instead we penalize 95% of the misfits in an L2 Gaussian way,
hoping them all to become much smaller
than the remaining L1 controlled residuals.
That leaves 5% of the time values to take advantage
of the weak pressure they will feel from L1
so this 5% can make the jumps between blocks.

-----------------------------------------------

QUESTION:   Is there a horse (search direction) named IRLS?
QUESTION:   How does IRLS relate to the strategy we plan?

ANSWER:  No.
ANSWER:  IRLS descends using a weighted L2 estimate of distance.
         We will descend using the exact L1 criterion (weighted median).