Iterative methods (like conjugate-directions) can sometimes be accelerated by a change of variables. The simplest change of variable is called a ``trial solution''. Formally, we write the solution as
(1) |
(2) |
(3) |
We hope this change of variables has saved effort. For each iteration, there is a little more work: Instead of the iterative application of and we have iterative application of and .Our hope is that the number of iterations decreases because we are clever, or because we have been lucky in our choice of .Hopefully, the extra work of the preconditioner operator is not large compared to .If we should be so lucky that ,then we get the solution immediately. Obviously we would try any guess with .Where I have known such matrices, I have often found that convergence is accelerated, but not by much. Sometimes it is worth using for a while in the beginning, but later it is cheaper and faster to use only .A practitioner might regard the guess of as prior information, like the guess of the initial model .
For a square matrix ,the use of a preconditioner should not change the ultimate solution.
Taking to be a wide rectangular matrix,
reduces the number of adjustable parameters,
changes the solution,
gets it quicker, but lower resolution.