# Optimization Solvers Basics

Classical optimization methods are linear methods based on calculation of a direction and search for the best point in the direction of the chosen gradient. The direction is calculated for each iteration. Examples of these methods are Conjugate Gradients and Quickest Descent. The search method usually requires calculation of the objective function approximation in a neighborhood of the current point.

Optimization problems generally have three basic components:

- An objective function that needs to be minimized or maximized. For instance, in a manufacturing process, you might want to maximize the profit or minimize the cost. In fitting experimental data to a user-defined model, the total deviation of observed data can be minimized from predictions based on the model.

- A set of unknowns or variables that affect the value of the objective function. In the manufacturing problem, the variables might include the amount of resources used or the time spent on each activity. In fitting-the-data problem, the unknowns are the parameters that define the model.

- A set of constraints that allow unknowns to take on certain values but exclude others. In the manufacturing problem, it does not make sense to spend a negative amount of time on any activity, so all the "time" variables are constrained to be non-negative.

The overall goal is to find values of the variables that minimize or maximize the objective function while meeting the constraints.

Intel® MKL provides math optimization tools, namely the optimized Trust-Region (TR) solvers for nonlinear least squares problems with or without linear constraints.