\cleardoublepage
\thispagestyle{septwo}
\label{ben1.start}
\def\papername{ben1}
\def\figdir{./Fig}
%\shortnote

\lefthead{Witten and Grant}
\righthead{Convex Optimization}
\footer{SEP-125}
\email{ben@sep.stanford.edu,mcgrant@stanford.edu}
\title{Interval velocity estimation through convex optimization}
\author{Ben Witten and Michael Grant}
\ABS{Convex optimization is an optimization technique which maximizes
  efficiency by fully harnessing the convex nature of certain
  problems.  Here we test a convex optimization solver on a least-squares formulations
  of the Dix equation.  Convex optimization has many
  useful traits including the ability to set bounds on the solution
  which are explored here.  As well, this example
  serves as a test for the feasibility of convex optimization for
  future, more expensive tomography problems.   
}
%
\section{Introduction}
%
Interval velocity estimation is a fundamental problem in seismology.
The simplest technique for finding interval velocity is the Dix
equation \cite{dix} which analytically inverts the root-mean-square
(RMS) velocity for interval velocity.  The Dix equation has many
flaws including the assumption of a vertically stratified earth and
numerical problems that can cause the inversion to become unstable for
rapidly varying velocities.  To better constrain the solution, the Dix
equation is often cast as a least-squares problem, which is
regularized in time with a differential operator that penalizes rapid
variations to produce a smooth result \cite{Clapp.sep.97.bob1}. 
%
\par
%
\longcite{Valenciano.sep.114.alejandro1} expanded on this work to use
both $\ell_2$ and $\ell_1$ regularization.  The $\ell_2$ regularization is
justified when the expected velocity model is smooth.  When geological
expectations dictate abrupt changes in interval velocity $\ell_1$
regularization can be utilized to preserve sharp boundaries when they
are present, yet allows for smooth velocity elsewhere.
%
\par
%
Since least-squares problems are already convex, this is a perfect
problem to test the utility of convex optimization.  Here we
utilize convex optimization to solve the problem of
interval velocity estimation using the same examples as those present
by \longcite{Valenciano.sep.114.alejandro1} for the $\ell_2$ and
$\ell_1$ regularization, which used conjugate gradient methods.  As well, bounds will
be enforced on the solution to further constrain the Dix inversion to
a more geologically sensible answer.
%
\section{Least-Squares Dix Equation}
%
The Dix equation is a nonlinear relationship between RMS velocity and
interval velocity.  It is, however, linear in the square of the interval
velocity.  This linearized formulation of the Dix equation was solved by
\longcite{Clapp.sep.97.bob1} by using a preconditioned least-squares
  optimization with spatial smoothness constraints.  In this  approach
  or data fitting goal is to minimize the residual of 
\begin{equation}
\bf  W(Cu-d) \approx 0\label{ben1/eq:ls}
\end{equation}
where $\bf u$ is a vector whose components range over vertical
traveltime depth $\tau$ and whose values are the interval velocity
squared $v^2_{int}$.  $\bf d$ is the data vector which has the same
range as $\bf u$, but whose values are the scaled RMS velocity squared
$\tau v^2_{RMS}/\Delta\tau$ where $\tau/\Delta\tau$ is the index on
the time axis.  $\bf C$ is the casual integration operator.  And $\bf W$ is a weight matrix which is proportional to
our confidence in RMS velocities. 
%
\par
%
Since the fitting goal, equation \ref{ben1/eq:ls}, is unstable when there are high
frequency variations in RMS velocity, a regularization term is added
to penalize this erratic behavior.  As done by
\longcite{Valenciano.sep.114.alejandro1}, first order derivatives are
used.  This system of equations is
\begin{eqnarray}
        \bf W ( Cu - d ) \approx 0  \label{ben1/eq:ls2} \nonumber\\
    \epsilon_{\tau} { \bf D_{\tau} u} { \bf \approx 0 }  \nonumber
\\
    \epsilon_{x}    { \bf D_x u}      { \bf \approx 0 } \label{ben1/eq:ls2}
\end{eqnarray}
where ${ \bf D_{\tau} }$ and ${ \bf D_x }$ are first-order finite differences
derivatives in traveltime and midpoint, respectively.
$\epsilon_{\tau}$ and $\epsilon_{x}$ balance the relative importance
of the two model residuals, effectively controlling the smoothness.
%
\par
%
The approach taken towards the regularization terms will determine
whether a smooth or discontinuous model is found.  $\ell_2$
regularization will produce a smooth result.  If a discontinuous
velocity is geologically expected, such as for carbonates, $\ell_1$
regularization can be used to produce a blocky model \cite{Valenciano.sep.114.alejandro1}.
%
\section{Convex Optimization}
%
A problem is a convex optimization problem if it has the form
\begin{eqnarray}
\mbox{minimize } f_0(x)~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ \nonumber\\
\mbox{subject to } f_i(x) \leq b_i,\mbox{ } i=1,\ldots,m 
\end{eqnarray}
where $f_0,\ldots,f_m$ and $b_1,\ldots,b_m$ are convex functions and $x$ exists in a convex
set, $\bf S$ \cite{boyd}. Convex optimization problems have many
attractive features including the guarantee of local minima to be
global and strong optimality, feasibility, and
sensitivity information.  As well, there are reliable and efficient numerical
algorithms to solve these problems.  Since this Dix formulation is a
least-squares problem, which is 
already convex, it seems natural to try convex optimization techniques
to find the solution.  
%
\par
%
The data fitting goal, equation (\ref{ben1/eq:ls}), can be rewritten in optimization
notation as 
\begin{equation}
\mbox{minimize } \| \bf{W(Cu-d)}\|_2
\end{equation}
where $\| \cdot \|_2$ means the least-squares norm.  When necessary
regularization terms are added the full set of goals, equation (\ref{ben1/eq:ls2}) becomes
\begin{equation}
\mbox{minimize } \|\bf{W(Cu-d)}\|_2+\|\bf \epsilon_{\tau} D_{\tau}u\|_i+\|\bf \epsilon_{x} D_{\tau} u\|_i\label{ben1/eq:cv1}
\end{equation}
in optimization notation.  If $i$ is $2$ then an $\ell_2$ regularization is used and a smooth
model is obtained.  If  $i$ is $1$ then an $\ell_1$ regularization
is used and the result is a blocky model, instead.  
%
\par
%
Even after inversion, there may be points in the model
space which do not make geological sense, usually due to picking
errors caused by poor resolution.  Convex optimization allows for bound constraints to be
imposed on the solution, which can correct for such inconsistencies.  
If we constrain the solution then we have
\begin{eqnarray}
\mbox{minimize } \|\bf{W(Cu-d)}\|_2+\|\bf \epsilon_{\tau} D_{\tau} u\|_{\rm{i}}+\|\bf
\epsilon_{x} D_{\tau} u\|_{\rm{i}} \nonumber \\
u \leq v_{\rm{max}} ~~~~~~~~~~~~~~~~~~~~~~~~~\nonumber \\
u \geq v_{\rm{min}}, ~~~~~~~~~~~~~~~~~~~~~~~~~\label{ben1/eq:cv2}
\end{eqnarray}
where $ v_{\rm{max}}$ and $v_{\rm{min}}$ are the square of the
maximum and minimum allowable velocity models, respectively.
%
\par
%
To do the convex optimization, $\bf cvx$ \cite{grant} will be
used.  $\bf cvx$ is a MATLAB based system for solving convex optimization
problems.  It allows constraints and objectives to be specified with
common MATLAB syntax.
%
\section{Real Data Examples}
%
125 CMP's from a 2-D prestack data set that was acquired in the Gulf
of Mexico were used.  Since the Gulf of Mexico usually exhibits flat
reflectors, this is suitable for the Dix equation.  The region is also
faulted which implies discontinuous velocities.
%
\par
%
The approach taken for obtaining the RMS velocities is the same as
that in \cite{Valenciano.sep.114.alejandro1}, but will briefly be recreated
here.  First, velocity analysis was preformed on each CMP.  Then an
auto-picker was used to pick the maximum stacking power corresponding
to the best RMS velocity at each CMP position.  An example velocity
analysis with picks for a single CMP is shown on the top of Figure \ref{ben1/fig:vel-vrms}.  The
velocity values from all the CMPs can be combined to form a complete
RMS velocity model space.  This is shown on the bottom of Figure \ref{ben1/fig:vel-vrms}.
Please note that the velocities picked in Figure \ref{ben1/fig:vel-vrms} are in
slowness rather than velocity, while the complete RMS model space is
in velocity.  This is because the conjugate gradient method used a
slowness model, but the convex optimization failed with slowness and
velocity had to be used.  This limitation may be because the values were all
close to $0$ forcing the solution down or due to the narrow range of
slowness values.  Thus the images computed with conjugate gradients
were computed in slowness and inverted to velocity, while the convex
optimization images were computed directly with velocity values.
\activeplot{vel-vrms}{width=4.5in}{}{Top: Auto-picked RMS velocity of one CMP from a
  2-D prestack dataset. Bottom: Raw RMS velocity map for all 125 CMPs}
Figure \ref{ben1/fig:stack1} shows the stacked section which displays the
faulting mentioned above.  The middle panel has the faults highlighted and the bottom
panels shows the same faults on the raw RMS velocity.  It is
interesting to note that the raw RMS velocity shows the faults clearly. 
\activeplot{stack1}{width=5.0in,height=8.0in}{}{Top: Stacked data using the raw
  RMS velocity.  Middle: Stack data with lines showing the faults.
  Bottom: RMS velocity with same lines showing the faults.}
%
\par
%
The top of Figure \ref{ben1/fig:L2vint} shows the interval velocity resulting from
solving equations (\ref{ben1/eq:ls2}) with $\ell_2$ regularization.  The
bottom of Figure \ref{ben1/fig:L2vint} shows the interval
velocity when equation (\ref{ben1/eq:cv1}) is solved with  $\ell_2$
regularization.  Note that all interval velocities are clipped at
their respective maximums.
Both images in Figure \ref{ben1/fig:L2vint} are very similar showing that convex optimization is at
least equivalent to conjugate gradients in terms of quality of
solution.  
\activeplot{L2vint}{width=4.5in}{}{Top: Interval velocity computed using conjugate
  gradients with $\ell_2$ regularization. Bottom: Interval velocity computed using convex optimization with $\ell_2$ regularization.} 
%
\par
%
If we now look at the solutions to equations (\ref{ben1/eq:ls2})
and (\ref{ben1/eq:cv1}) solved with a $\ell_1$ regularization, shown in
Figure \ref{ben1/fig:L1vint}, we can see that, as expected, a much blockier solution is
found.  As in the previous figure, 
the top panel of Figure \ref{ben1/fig:L1vint}, created by conjugate gradients, is very
similar to the bottom image, solved with convex optimization.
\activeplot{L1vint}{width=4.5in}{}{Top: Interval velocity computed using
  conjugate gradients with $\ell_1$ regularization. Bottom: Interval velocity computed using convex
  optimization with $\ell_1$ regularization.} 
%
\par
%
In the $\ell_1$ regularization image, the faults do show up faintly.
If we overlay the same lines shown in Figure \ref{ben1/fig:stack1} onto
Figure \ref{ben1/fig:L1vint}, this becomes more obvious as shown in Figure
\ref{ben1/fig:L1-interp}.  The faults may be
slightly more obvious in the problem solved with $\bf cvx$, which is
blockier than with conjugate gradients.  The difference in
``blockiness'' is due to the different $\epsilon$'s and how they are
applied in each case.
 \activeplot{L1-interp}{width=4.5in}{}{Same as Figure \ref{ben1/fig:L1vint} except
 with lines marking fault locations.} 
%
\par
%
There appears to be some low velocity anomalies near the bottom of the
interval velocity solution.  This is predominately seen in the blocky
models, but there are uncharacteristically low velocities at late times
in all the models.  As we can see from the stack in Figure
\ref{ben1/fig:stack1}, there is no evidence to support such velocities.  To correct this we can constrain the solution
further by adding bounds when solving the convex optimization
problem.  If we assume that the interval velocity $v(z)$ increases
linearly with depth:
\begin{equation}
v(z)=v_0+\alpha z,
\end{equation} 
then we can get a general estimate of the interval velocity.  This is
shown in Figure \ref{ben1/fig:bounds}.  Velocities 20 percent above and below
this model are used as the maximum and minimum constraints in equation
(\ref{ben1/eq:cv2}). Figures \ref{ben1/fig:L2vint-cvx-bounded} and \ref{ben1/fig:L1vint-cvx-bounded}
show the bounded constrained $\ell_2$ and $\ell_1$ solutions,
respectively.  As we can see, the low velocities occurring at late times
have been attenuated.  
\activeplot{bounds}{width=4.5in}{}{Model from which the upper and lower
    velocity constraints are formed.  Upper is $20\%$ greater than this
    everywhere and the lower is $20\%$ less.}
\activeplot{L2vint-cvx-bounded}{width=4.5in}{}{Solution by convex optimization with
  $\ell_2$ regularization and bound constraints.}
\activeplot{L1vint-cvx-bounded}{width=4.5in}{}{Solution by convex optimization with
  $\ell_1$ regularization and bound constraints.}
%
\section{Conclusions}
%
Convex optimization methods show promise for solving
least-squares problems.  As exemplified by the least-squares Dix
equation, convex optimization can yield similar results to those
obtained through conjugate gradient methods.  Yet convex optimization
also has the advantage of imposing geologically constrained bounds to
further enhance the solution.  The $\ell_1$ $\bf cvx$ solution also shows
a slightly better ability to pick up the faults than the conjugate
gradient method.  As stated above, this could simply be a
function of $\epsilon$ choice.
%
\par
%
The convex optimization solver may not be as fast as conjugate
gradient methods, but the solution obtained is guaranteed to be
correct.  The conjugate gradient solution is usually obtained by
iterating until we are tired.  The convex solver, on the other hand,
works until a preset accuracy is achieved.  In these problems, this
precision was set at $10^{-9}$.  The efficiency of $\ell_2$
versus $\ell_1$ regularization is quite striking.  It takes $8$ times
more iterations to do the $\ell_1$ than the $\ell_2$, but only $3$
times as many when both are bounded.  This is difference comparable to
that of conjugate gradients.
%
\par
%
To apply convex optimization larger problems and more complex
operators, a convex optimization solver that does not rely on MATLAB
is needed.  While the $\bf cvx$ software is efficient and easy to use, it is
limited by MATLAB's efficiency and lack of memory.  If a new solver
can be created convex optimization could be successful for future endeavors.
%
\section{Acknowledgments}
%
We would like to thank Bob Clapp for his discussion and advice on this
project and Biondo Biondi for bringing us together to attempt convex
optimization on SEP problems.
\bibliographystyle{sep}
\bibliography{SEP,MISC,GEOTLE,../ben1/refs1}
\clearpage
\label{ben1.end}
