How do you minimize and maximize #f(x,y)=(x-y)(x+y)+sqrt(xy)# constrained to #0<xy-y^2<5#?

We will searching for stationary points, qualifying then as local maxima/minima.

First we will transform the maxima/minima with inequality restrictions into an equivalent maxima/minima problem but now with equality restrictions.

To do that we will introduce the so called slack variables #s_1# and #s_2# such that the problem will read.

Maximize/minimize #f(x,y) = (x - y) (x + y) + sqrt[x y]#
constrained to

#{ (g_1(x,y,s_1)=x y - y^2 - s_1^2=0), (g_2(x,y,s_2)=x y - y^2 + s_2^2 - 5=0) :}#

The lagrangian is given by

#L(x,y,s_1,s_2,lambda_1,lambda_2) = f(x,y)+lambda_1 g_1(x,y,s_1)+lambda_2g_2(x,y,s_2)#

The condition for stationary points is

#grad L(x,y,s_1,s_2,lambda_1,lambda_2)=vec 0#

so we get the conditions

#{ (2 x + lambda_1 y + lambda_2 y + y/(2 sqrt[x y]) = 0), (lambda_1 (x - 2 y) + llambda_2 (x - 2 y) - 2 y + x/(2 sqrt[x y]) = 0), (-s_1^2 + x y - y^2 = 0), (-2 lambda_1 s_1 = 0), (-5 + s2^2 + x y - y^2 = 0), (2 lambda_2 s_2 = 0) :}#

Solving for #{x,y,s_1,s_2,lambda_1,lambda_2}# we have

#{(x= -5.15905, y= -3.86559, lambda_1 = 0., s_1 = -2.23607, lambda_2= -2.78118, s_2= 0.), (x = 5.15905, y = 3.86559, lambda_1= 0., s_1 = -2.23607, lambda_2= -2.78118, s_2= 0.) :}#

The restriction #g_2(x,y)# is active (#lambda_2 ne 0# and #s_2=0#) so for qualifying the maxima/minima we produce

#f_{g_2}(x) = 1/2 (10 + sqrt[2] sqrt[x (x - sqrt[-20 + x^2])] pm x (x + sqrt[x^2-20]))#

Computing

#d/(dx)(f_{g_2}( pm5.15905)) = 0#

and

#d^2/(dx^2)(f_{g_2}( pm5.15905)) = 1.64412#

we conclude that the found solutions are local minima points.