MEDIANS AND REGRESSION

Let r_i be components of a residual vector $\bold r$and consider perturbation components $\Delta r_i$.We study the properties of the new residual

$$\tilde {\bold r} \quad =\quad\bold r +\alpha \Delta \bold r$$ (1)

where $\alpha$ is a scaling factor. Consider the implications of

$$\begin{eqnarray} 0 &=& \mbox{median}_i \left( { \tilde r_i \over \Delta r_i} \ri... ...t) = \mbox{median}_i \left( { r_i\over \Delta r_i}\right) +\alpha\end{eqnarray}$$ (2) (3)

Thus choosing $\alpha=-\mbox{median}_i ( r_i / \Delta r_i)$ causes the median of $\tilde r_i /\Delta r_i$ to vanish which means $\tilde r_i /\Delta r_i$ has as many positive terms as negative ones. Consequently, the number of polarity agreements in the component pairs $(\tilde r_i,\Delta r_i)$of the new residual $\tilde {\bold r}$and the regressor $\Delta \bold r$ equals their polarity disagreements. In other words,

$$0 \quad =\quad\sum_i \mbox{sgn}(\tilde r_i)\;\mbox{sgn}(\Delta r_i) \quad =\quad\mbox{agreements} - \mbox{disagreements}$$ (4)

In what sense have we created the smallest new residual $\tilde {\bold r}$? It is smallest in the sense that if we change $\tilde {\bold r}$by adding or subtracting some $\epsilon$of the perturbation $\epsilon\Delta \bold r$,the count of growing components less the decreasing components of $\tilde {\bold r} +\epsilon \Delta \bold r$ is

$$\sum_i \mbox{sgn}(\tilde r_i+\epsilon \Delta r_i)\;\mbox{sgn}(\epsilon\Delta r_i) \quad \ge \quad 0$$ (5)

The best residual is one in which any perturbation serves only to increase the polarity agreements with the regressor. We chose $\alpha$ to reduce as many components of $\tilde {\bold r}$ as we could.