fminunc stopped because it cannot decrease the objective function

Dear all,

This is my main code. this error shows " fminunc stopped because it cannot decrease the objective function". i could not solve it..

%% Tutorial#5 - Implement Logistic Regression
% Clear all variables and close all plots
clear all; close all; 

%% ******************* Loading Data **********************
data = load('dataset.data');
disp('The dataset was loaded sucessfully!');

X = data(:,1:end-1);% features
y = data(:,end);% class labels

disp('Press any key to continue.');
pause;

%% ****************** Calculate Cost and Gradient *******************
X = [ones(size(X,1),1) , X];
[cost, grad] = calculateCost(zeros(size(X,2),1), X, y);
disp('The cost when theta values initialized to zeros');
disp(cost);

%% ******************* Learn Theta Values *******************
%  Set options for fminunc
options = optimset('GradObj', 'on', 'MaxIter', 400);

%  This function will return the learned theta values and the cost 
[theta, cost] = ...
	fminunc(@(t)(calculateCost(t, X, y)), zeros(size(X,2),1), options);

disp('Theta values found by fminunc');
disp (theta);
==================================================================
%% The function compute the cost and gradient of a logistic regression model 
% Parameters:
% X is the matrix of the features for all the examples (m)
% y is the vector of the targets for all the examples (m)
% theta is a column vector of all theta values

function [J, grad] = costFunction(theta, X, y)

%COSTFUNCTION Compute cost and gradient for logistic regression
%   J = COSTFUNCTION(theta, X, y) computes the cost of using theta as the
%   parameter for logistic regression and the gradient of the cost
%   w.r.t. to the parameters.

% Initialize some useful values
m = length(y); % number of training examples
J = 0;

grad = zeros(size(theta));


% calculate cost function
h = sigmoid(X*theta);
J = ((-y)'*log(h)-(1-y)'*log(1-h))/m;
% calculate grads
grad = (X'*(h - y))/m;

end
==============================================================
function g = sigmoid(z)
%SIGMOID Compute sigmoid functoon
%   J = SIGMOID(z) computes the sigmoid of z.
g = zeros(size(z));
g = 1./(1+ exp(-z));

end

Answer 1

[Sai Bhargav Avula]

Hi,

fminunc is a derivative based optimizer. If the objective function has discontinuities or have multiple optimums then is sensitive to the initialization. I would recommend to have random initial points rather ZEROS also try using multistart or pattersearch.

Answer 2

[Matt J]

The log of a sigmoid is a numerically delicate operation. You need to write your objective function with a dedicated log-sigmoid function as described here:

https://www.mathworks.com/matlabcentral/answers/456670-fminunc-stopped-because-it-cannot-decrease-the-objective-function-along-the-current-search-direction#comment_694918