Calculus Optimization Methods/Lagrange Multipliers

From Wikibooks, the open-content textbooks collection

Jump to: navigation, search

The method of Lagrange multipliers solves the constrained optimization problem by transforming it into a non-constrained optimization problem of the form:

  • \operatorname{\mathcal{L}}(x_1,x_2,\ldots, x_n,\lambda)= \operatorname{f}(x_1,x_2,\ldots, x_n)+\operatorname{\lambda}(k-g(x_1,x_2,\ldots, x_n))

Then finding the gradient and hessian as was done above will determine any optimum values of \operatorname{\mathcal{L}}(x_1,x_2,\ldots, x_n,\lambda).

Suppose we now want to find optimum values for f(x,y) = 2x2 + y2 subject to x + y = 1 from [2].

Then the Lagrangian method will result in a non-constrained function.

  • \operatorname{\mathcal{L}}(x,y,\lambda)= 2x^2+y^2+\lambda (1-x-y)

The gradient for this new function is

  • \frac{\partial \mathcal{L}}{\partial x}(x,y,\lambda)= 4x+\lambda (-1)=0
  • \frac{\partial \mathcal{L}}{\partial y}(x,y,\lambda)= 2y+\lambda (-1)=0
  • \frac{\partial \mathcal{L}}{\partial \lambda}(x,y,\lambda)=1-x-y=0

Finding the stationary points of the above equations can be obtained from their matrix from.

\begin{bmatrix} 4 & 0 & -1 \\ 0& 2 & -1 \\ -1 & -1 & 0 \end{bmatrix} \begin{bmatrix} x\\ y \\ \lambda \end{bmatrix}= \begin{bmatrix} 0\\ 0\\ -1 \end{bmatrix}

This results in x = 1 / 3,y = 2 / 3,λ = 4 / 3.

Next we can use the hessian as before to determine the type of this stationary point.

H(\mathcal{L})= \begin{bmatrix} 4 & 0 & 0 \\ 0& 2 & 0 \\ 0&0&0 \end{bmatrix}

Since H(\mathcal{L}) >0 then the solution (1 / 3,2 / 3,4 / 3) minimizes f(x,y) = 2x2 + y2 subject to x + y = 1 with f(x,y) = 2 / 3.

Retrieved from "http://en.wikibooks.org/wiki/Calculus_Optimization_Methods/Lagrange_Multipliers"