Second step of the improvement

Now let us assume n > 2 and add some amount of the step from the (n-2)-th iteration to the search direction, determining the new direction ${\bf s}_n^{(n-2)}$, as follows:  

$${\bf s}_n^{(n-2)} = {\bf s}_n^{(n-1)} + \beta_n^{(n-2)}\,{\bf s}_{n-2}\;.$$ (17)

We can deduce that after the second change, the value of numerator in equation (9) is still the same:  

$$\left({\bf r}_{n-1},\,{\bf A\,s}_n^{(n-2)}\right)^2 = \left[... ...ight)\right]^2 = \left({\bf r}_{n-1},\,{\bf A\,c}_n\right)^2\;.$$ (18)

This remarkable fact occurs as the result of transforming the dot product $\left({\bf r}_{n-1},\,{\bf A\,s}_{n-2}\right)$ with the help of equation (4):  

$$\left({\bf r}_{n-1},\,{\bf A\,s}_{n-2}\right) = \left({\bf r... ..._{n-1}\,\left({\bf A\,s}_{n-1},\,{\bf A\,s}_{n-2}\right) = 0\;.$$ (19)

The first term in (19) is equal to zero according to formula (7); the second term is equal to zero according to formula (15). Thus we have proved the new orthogonality equation  

$$\left({\bf r}_{n-1},\,{\bf A\,s}_{n-2}\right) = 0\;,$$ (20)

which in turn leads to the numerator invariance (18). The value of the coefficient $\beta_n^{(n-2)}$ in (17) is defined analogously to (14) as  

$$\beta_n^{(n-2)} = - {{\left({\bf A\,s}_n^{(n-1)},\,{\bf A\,... ...bf A\,s}_{n-2}\right)} \over {\Vert{\bf A\,s}_{n-2}\Vert^2}}\;,$$ (21)

where we have again used equation (15). If ${\bf A\,s}_{n-2}$ is not orthogonal to ${\bf A\,c}_n$, the second step of the improvement leads to a further decrease of the denominator in (8) and, consequently, to a further decrease of the residual.