Normal equations

The basic least-squares equations are often called the ``normal" equations. The word ``normal" means perpendicular. We can rewrite equation (16) to emphasize the perpendicularity. Bring both terms to the left, and recall the definition of the residual $\bold r$from equation (2):

$$\begin{eqnarray} \bf B' ({\bf d}- \bf B \bf x ) &=& \bold 0 \ \bf B' \bold r &=& \bold 0\end{eqnarray}$$ (23) (24)

Equation (24) says that the residual vector $\bold r$is perpendicular to each row in the $\bf B'$ matrix. These rows are the fitting functions. Therefore, the residual, after it has been minimized, is perpendicular to the fitting functions.