		Alternatives to cgstep?
		----------------------

Recently the TDF class analyzed 4 data sets.
3 of 4 had noise bursts in them and the 4-th had spikes
removed (somehow) before we got the data.

This experience gave me new respect for Iterated ReWeighted LS.
Thus a hasty 5 page addition to TDF (see my web home page).
It also led me to many vague questions as well as some very
specific questions about whether I can improve cgstep().

First I remark that PVI invokes cgstep() in 15 applications
and TDF invokes it in 28 applications, and also, cgstep()
is heavily used by jun, dimitri, ray, matt, and sometimes david
all who have many applications of their own.  I mention these many
uses because I hope changes will affect all in a favorable way
when all applications are made optionally robust by
a straightforward change in the "fortran CG user template".

I already changed the polarity of the residual to r=Fm-d
in a new conjugate-gradient subroutine cgplus.
I changed nonlinear and big-model-space solvers
so that they will accept a starting guess.

Question:
Observed bad solver behavior on nonlinear problems needs to be addressed.
Will the cgtemp.rt subroutine climb over the difficulty
or do we need something better?

--------------
|Revising weights with each iteration:
|	
|We have seen examples where it was dangerous
|to revise weights from one iteration to the next.
|Much more understanding is needed.
|It might be wiser to leaky integrate past inverse weights.
|
|	wtinv =  .9 * wtinv +			# stabilize over iterations
|		        |r| +			# throw out spikes
|			RunningMedian(|r|)	# reliable variance estimate
|		    or 	RunningMean(  |r|)	# a variance estimate
|
|All these running medians could be slow.
|Also need think about color of residuals.


------------------------------

Some long term issues:
 - Both templates unnecessarily square the condition number.
 - We still see mystery programs that don't behave.
 - People don't like Saunders LSQR, they can't snapshot the process.
 - Want speed:  preconditioning, grid refinement, multiscale
 - Some say to precondition only early iterations.
Review some nonlinear issues.
 - Early iterations must not be allowed to go wild.
 - Users must be able to view and guide the descent in ad hoc ways.
 - Possible theory?

