Induction

Continuing by induction the process of adding a linear combination of the previous steps to the arbitrarily chosen direction ${\bf c}_n$(known in mathematics as the Gram-Schmidt orthogonalization process), we finally arrive at the complete definition of the new step ${\bf s}_n$, as follows:  

$${\bf s}_n = {\bf s}_n^{(1)} = {\bf c}_{n} + \sum_{j=1}^{j=n-1}\,\beta_n^{(j)}\,{\bf s}_{j}\;.$$ (22)

Here the coefficients $\beta_n^{(j)}$ are defined by equations  

$$\beta_n^{(j)} = - {{\left({\bf A\,c}_n,\,{\bf A\,s}_{j}\right)} \over {\Vert{\bf A\,s}_{j}\Vert^2}}\;,$$ (23)

which correspond to the orthogonality principles  

$$\left({\bf A\,s}_n,\,{\bf A\,s}_{j}\right) = 0\;,\;\;1 \leq j \leq n-1$$ (24)

and  

$$\left({\bf r}_{n},\,{\bf A\,s}_{j}\right) = 0\;,\;1 \leq j \leq n\;.$$ (25)

It is these orthogonality properties that allowed us to optimize the search parameters one at a time instead of solving the n-dimensional system of optimization equations for $\alpha_n$ and $\beta_n^{(j)}$.