Find minimum error function using gradient descent

Find the minimum error function using gradient descent

for the function (x-1).^2 - 2

I found a code in matlabcentral, but I'm not sure how to edit it according to my requirement.

%% Code I got from matlabcentral
X = -2:0.1:2;
Y = -2:0.1:2;
[X,Y] = meshgrid(X,Y);
% Z = 2*X.^2+3*Y.^2;
Z = (x-1).^2 - 2;
surf(X,Y,Z)
hold on
x(1) = 2; % initial value of x 
y(1) = 2; % initial value of y 
z(1) =  2.*x(1).^2 + 3.*y(1).^2;
stepsize = 0.1;
for i = 1:30
    zx = 4*x(i); 
    zy = 6*y(i); 
    x(i+1) = x(i) - stepsize*zx; %gradient descent 
    y(i+1) = y(i) - stepsize*zy;
    z(i+1) = 2.*x(i+1).^2 + 3.*y(i+1).^2
end

Answer 1

[Dyuman Joshi]

I don't know about the code you found, the method of operation for the given optimization algorithm should be like this -

funcChoice=2;
%error function
switch funcChoice
    case 1

        fun = @(x) (x-1).^2 - 2;
        fungrad = @(x) 2*(x-1);
        minval = -2;
    case 2
        
        fun = @(x) exp(x) - 2*x;
        fungrad = @(x) exp(x)-2;
        minval = 2*(1-log(2));
    case 3
        
        fun = @(x) -log(x) + 2*x;
        fungrad = @(x) -1./x+2;
        minval = 1+log(2);
end

disp(fun)

    @(x)exp(x)-2*x

disp(fungrad)

    @(x)exp(x)-2

disp(minval)

    0.6137

%starting point, can be changed as per wish
x=4;
%learning rate
rate = 0.01;
%counter
itr=0;
%tolerance, adjust accordingly
tol=1e-3;

%main algorithm
while abs(fun(x(end))-minval)>=tol
    itr=itr+1;
    x(end+1)=x(end)-rate*fungrad(x(end));
end

%number of iterations
itr

itr = 169

%corresponding x-value, already had a starting point, that's why +1 
finalx=x(itr+1)

finalx = 0.7242

%minimum function value
finalval=fun(x(itr+1))

finalval = 0.6147

%minimum error
minError=abs(minval-finalval)

minError = 9.7524e-04

[Dyuman Joshi]

1 - Yes, the corresponding minimum value is finalx.

3 - No, the gradient is different for all functions. The gradient is equal to d(func)/dx (for single variable functions, which is the case here). You will have to do that manually, and define them according to each function

2 - That is done using calculus. A simplified explaination -

Find values of x that satisfy d(func)/dx = 0. Now, calculate d2(func)/dx2 (second derivative) and plug values found from first equation. If any of the output comes out negative, that point is a minima.

Minima and Maxima by Calculus

Also, natural logarithm i.e. ln(x) is defined as log(x) in MATLAB

[Dyuman Joshi]

I have edited my answer accordingly, please check it.

[Dyuman Joshi]

If my answer helped you solve the problem, please accept it!

[Elysi Cochin]

@Dyuman Joshi Sir I need one more help related to this question.

I starting point for the first function, was given to me. I need to choose a good starting point for the other two functions.

Please can you tell me how I can choose a good starting point? I tried values like 1 to 5. But how can I know that which value is a good starting point? What can be the range of values I can give for starting point?

[Dyuman Joshi]

I don't know exactly what is expected from you. 'Good' starting point is subjective.

What was the starting point of the first function? That might give us some hint to think about the others.

[Elysi Cochin]

What they gave as good starting point of the first function was 4.

Any hint Sir?

[Dyuman Joshi]

That doesn't make any sense to me or doesn't strike any logic

Was a justification given for it? If not, check notes or ask the instructor what does it mean by a good point.

[Elysi Cochin]

Ok Sir. Thank you so much for your help. Thanks a lot.