yawiki.org entry for Lagrange multipliers

	Navigation Home Recent Most Active Popular Blog Credits RSS
	Interaction Register Statistics
	Help Suggestions Contact Us How to Edit Help

[Edit]

In mathematical optimization problems,
Lagrange multipliers, named after Joseph Louis Lagrange, is a method for finding the local extrema of a function of several variables subject to one or more constraints. This method reduces a problem in n variables with k constraints to a solvable problem in n + k variables with no constraints. The method introduces a new unknown scalar variable, the Lagrange multiplier, for each constraint and forms a linear combination involving the multipliers as coefficients.

The justification for this can be carried out in the standard way as concerns partial differentiation, using either total differentials, or their close relatives, the chain rules. The objective is to find the conditions, for some implicit function, so that the derivative in terms of the independent variables of a function equals zero for some set of inputs.

    Lagrange multipliers
        Introduction
        The method of Lagrange multipliers
            Very simple example
            Another example
        Economics
        See also

top

Introduction
Consider a two-dimensional case. Suppose we have a function, f(x,y), to maximize subject to

$gleft( x,y$

ight) = c,

where c is a constant. We can visualize contours of f given by

$f left( x, y$

ight)=d_n

for various values of

d_n

, and the contour of

g

given by

g ( x, y ) = c

. Suppose we walk along the

g = c

contour. Since, in general, the contours of

f

and

g

will be distinct, traversing the

g = c

contour will generally intersect and cross many contours of

f

. In general, by moving along the line

g = c

we can increase or decrease the value of

f

. Only when

g = c

, and the contour we are following, touches tangentially, but does not cross a contour of

f

, we do not increase or decrease the value of

f

. This occurs at the constrained local extrema and the constrained inflection points of

f

.

A familiar example can be obtained from weather maps, with their contours for temperature and pressure: the constrained extrema will occur where the superposed maps show touching lines (isopleths).

Geometrically we translate the tangency condition to saying that the gradients of

f

and

g

are parallel vectors at the maximum. Introducing an unknown scalar, λ, we solve

abla Bigf left(x, y ight) + lambda left(g left(x, y ight) - c ight) Big = 0
for λ ≠ 0.

Once values for λ are determined, we are back to the original number of variables and so can go on to find extrema of the new unconstrained function

$F left( x , y$

ight) =

f left( x , y 
ight) + lambda left( g left( x , y 
ight) - c 
ight)

in traditional ways. That is,

F(x,y) = f(x,y)

for all

(x , y)

satisfying the constraint because

g(x,y)-c

equals zero on the constraint, but the extrema of

F ( x ,  y )

are all on

g(x,y)=c

top

The method of Lagrange multipliers

Let f be a function defined on Rⁿ, and let the constraints be given by g_k(x) = 0 (perhaps by moving the constant to the left, as in g_k(x) − c = 0). Now, define the Lagrangian, Λ, as

$Lambda(mathbf x, oldsymbol lambda) = f + sum_k lambda_k g_k.$

Observe that both the optimization criteria and constraints g_k are compactly encoded as extrema of the Lagrangian:

abla_ Lambda = 0 Leftrightarrow abla_ f = - sum_k lambda_k abla_ g_k,

and

abla_ Lambda = 0 Leftrightarrow g_k = 0.

Often the Lagrange multipliers have an interpretation as some salient quantity of interest. To see why
this might be the case, observe that:

$rac = lambda_k.$

Thus, λ_k is the rate of change of the quantity being optimized as a function of the constraint variable.
As examples, in Lagrangian mechanics the equations of motion are derived by finding stationary points of the action, the time integral of the difference between kinetic and potential energy. Thus, the force on a particle due to a scalar potential, F = −∇V, can be interpreted as a Lagrange multiplier determining the change in action (transfer of potential to kinetic energy) following a variation in the particle's constrained trajectory. In economics, the optimal profit to a player is calculated subject to a constrained space of actions, where a Lagrange multiplier is the value
of relaxing a given constraint (e.g. through bribery or other means).

The method of Lagrange multipliers is generalized by the Karush-Kuhn-Tucker conditions.

top

Very simple example
Suppose you want to find the maximum value for

$f(x, y) = x^2 y$

with the condition that

$x^2 + y^2 = 1$

As there is just a single condition, we will use only one multiplier, say

lambda

$g (x, y) = x^2 +y^2 -1$

$Phi (x, y, lambda) = f(x,y) - lambda g(x, y) = x^2 y - lambda (x^2 + y^2 - 1)$

The maximum value is among the solution of the system of equations given by setting each of the partial derivatives of

Phi

equal to zero:

$2 x y - 2 lambda x = 0$

$x^2 - 2 lambda y = 0$

$x^2 + y^2 -1 = 0$

top

Another example
Suppose we wish to find the discrete probability distribution with maximal information entropy. Then

$f(p_1,p_2,ldots,p_n) = -sum_^n p_klog_2 p_k.$

Of course, the sum of these probabilities equals 1, so our constraint is g(p) = 1 with

$g(p_1,p_2,ldots,p_n)=sum_^n p_k.$

We can use Lagrange multipliers to find the point of maximum entropy (depending on the probabilities). For all k from 1 to n, we require that

$rac(f+lambda (g-1))=0,$

which gives

$racleft(-sum_^n p_k log_2 p_k + lambda (sum_^n p_k - 1)$

ight) = 0.

Carrying out the differentiation of these n equations, we get

$-left(rac+log_2 p_k$

ight) + lambda = 0.

This shows that all p_i are equal (because they depend on λ only).
By using the constraint ∑_k p_k = 1, we find

$p_k = rac.$

Hence, the uniform distribution is the distribution with the greatest entropy.

top

Economics

Constrained optimization plays a central role in economics. For example, the choice problem for a consumer is represented as one of maximizing a utility function subject to a budget constraint. The Lagrange multiplier has an economic interpretation as the shadow price associated with the constraint, in this case the marginal utility of income.

top

	Source Privacy License Download Contact Us Atlas Scientus.org Dictionary (Yet Another Wiki) RC : 1.39 MIT OpenCourseWare This article is licensed under the GNU Free Documentation License [copyleft]. It uses material from the Wikipedia article "Lagrange multipliers". link