Blocky models via the L1/L2 hybrid norm

Some convex functions and their derivatives

LEAST SQUARES:

$$C = r^2/2$$ (8)

$$C' = r$$ (9)

$$C'' = 1 \quad\quad \ge 0$$ (10)

L1 NORM:

$$C = \vert r\vert$$ (11)

$$C' = {\rm sgn}(r)$$ (12)

$$C'' = 0 {\rm or} \infty \quad\quad \ge 0$$ (13)

HYBRID:

$$C = R^2 (\sqrt{1 + r^2/R^2} - 1)$$ (14)

$$C' = {r\over\sqrt {1+r^2/R^2}}$$ (15)

$$C'' = \frac{1}{(1+ r^2/R^2)^{3/2}} \quad\quad \ge 0$$ (16)

HUBER:

$$C = \left\{ \begin{array}{l} \vert r\vert - R/2 \quad\quad {\rm i... ...\ge R \\ r^2/2R \quad\quad {\rm otherwise} \end{array}\right\}$$ (17)

$$C' = \left\{ \begin{array}{l} {\rm sgn} (r) \quad\quad {\rm if }... ...rt \ge R \\ r/R \quad\quad {\rm otherwise} \end{array}\right\}$$ (18)

$$C'' = \left\{ \begin{array}{l} 0 {\rm or} \infty \quad\quad {\rm... ...quad\quad {\rm otherwise} \end{array}\right\} \quad\quad \ge 0$$ (19)

I have scaled Hybrid so it naturally approaches the least squares limit as $R\rightarrow\infty$. As $R\rightarrow 0$, it tends to $C=R\vert r\vert$, scaled $L1$.

Because of the erratic behavior of $C''$ for $L1$ and Huber, and our planned use of second order Taylor series, we will not be using $L1$ and Huber norms here. Also, we should prepare ourselves for danger as HYBRID approaches the $L1$ limit. (I thank Mandy for reminding me of the infinite second derivative and I thank Mohammad and Nader for demonstrating numerical erratic behavior.)

Blocky models via the L1/L2 hybrid norm