Calculus Optimization Methods

From Wikibooks, the open-content textbooks collection

(Redirected from Calculus optimization methods)
Jump to: navigation, search
Applications-office.svg

This page was transwikied and needs to be de-wikified.
Wikibooks modules use wikilinks rather sparsely, and only to reference technical or esoteric terms that are critical to understanding the content. Most if not all wikilinks should simply be removed.
Please remove {{dewikify}} after the page is dewikified.
Nuvola apps bookcase.svg

A reader requests that this book be expanded to include more material.
You can help by adding new material (learn how) or ask for help in the reading room.

Contents

[edit] Introduction

A key application of calculus is in optimization: finding maximum and minimum values of a function, and which points realize these extrema.

[edit] Context

Formally, the field of mathematical optimization is called mathematical programming, and calculus methods of optimization are basic forms of nonlinear programming. We will primarily discuss finite-dimensional optimization, illustrating with functions in 1 or 2 variables, and algebraically discussing n variables. We will also indicate some extensions to infinite-dimensional optimization, such as calculus of variations, which is a primary application of these methods in physics.

[edit] Techniques

Basic techniques include the first and second derivative test, and their higher-dimensional generalizations.

A more advanced technique is Lagrange multipliers, and generalizations as Karush–Kuhn–Tucker conditions and Lagrange multipliers on Banach spaces.

[edit] Applications

Optimization, particularly via Lagrange multipliers, is particularly used in the following fields:

Further, several areas of mathematics can be understood as generalizations of these methods, notably Morse theory and calculus of variations.

[edit] Terminology

  • Input points, output values
  • Maxima, minima, extrema, optima
  • Stationary point, critical point; stationary value, critical value
  • Objective function
  • Constraints – equality and inequality
    • Especially sublevel sets
    • Feasible region, whose points are candidate solutions

[edit] Statement

This tutorial presents an introduction to optimization problems that involve finding a maximum or a minimum value of an objective function f(x_1,x_2,\ldots, x_n) subject to a constraint of the form g(x_1,x_2,\ldots, x_n)=k.

[edit] Maximum and minimum

Finding optimum values of the function f(x_1,x_2,\ldots, x_n) without a constraint is a well known problem dealt with in calculus courses. One would normally use the gradient to find stationary points. Then check all stationary and boundary points to find optimum values.

[edit] Example

  • f(x,y) = 2x2 + y2
  • fx(x,y) = 4x = 0
  • fy(x,y) = 2y = 0

f(x,y) has one stationary point at (0,0).

[edit] The Hessian

A common method of determining whether or not a function has an extreme value at a stationary point is to evaluate the hessian[3] of the function at that point. where the hessian is defined as

H(f)= \begin{bmatrix} \frac{{\partial}^2 f}{{\partial}^2 x_1} & \frac{{\partial}^2 f}{\partial x_1 \partial x_2} & \dots & \frac{{\partial}^2 f}{\partial x_1 \partial x_n} \\ \frac{{\partial}^2 f}{\partial x_2 \partial x_1} & \frac{{\partial}^2f}{{\partial}^2 x_2}& \dots & \frac{{\partial}^2f}{\partial x_2 \partial x_n}\\ \vdots & \vdots & \ddots & \vdots \\ \frac{{\partial}^2f}{\partial x_n \partial x_1} & \frac{{\partial}^2f}{\partial x_n \partial x_2}& \dots & \frac{{\partial}^2f}{{\partial}^2 x_n}\\ \end{bmatrix}.

[edit] Second derivative test

The Second derivative test determines the optimality of stationary point x according to the following rules [2]:

  • If H(f) > 0 at point x then f has a local minimum at x
  • If H(f) < 0 at point x then f has a local maximum at x
  • If H(f) has negative and positive eigenvalues then x is a saddle point
  • Otherwise the test is inconclusive

In the above example.

H(f)=\begin{bmatrix} 4 & 0\\ 0& 2 \end{bmatrix}.

Therefore f(x,y) has a minimum at (0,0).

[edit] Sections

[edit] References

[1] T.K. Moon and W.C. Stirling. Mathematical Methods and Algorithms for Signal Processing. Prentice Hall. 2000.
[2]http://www.ece.tamu.edu/~chmbrlnd/Courses/ECEN601/ECEN601-Chap3.pdf
Retrieved from "http://en.wikibooks.org/wiki/Calculus_Optimization_Methods"