Generalized-norm conjugate direction solver

Setting threshold with percentile

The benefit of a mixed $L_1/L_2$ norm is that small residuals can be optimized in an $L_2$ sense while large residuals are treated by $L_1$. This requires a definition of ``small residual''; how small is ``small''?

Our implementation addresses this issue with a numerical parameter, $r_t$. This is the threshold for transition between $L_1$ and $L_2$ in Huber and Hybrid norms. According to the analytic definition of each norm, $r_t$ adjusts the crossover point. Needless to say, pure $L_1$ and $L_2$ have no such transition. To reduce the problem-specific dependency of $r_t$, we have configured our solver to compute this threshold based on a user-defined percentile. By switching to percentile, we retain a physical meaning for this user-specified parameter. It is possible to use separate thresholds, and even different norms, for the model- and data-fitting goals of a general regularized or preconditioned optimization problem.

Generalized-norm conjugate direction solver