Simple Code working in the Command Line but not through function

I have been doing Andrew Ng's Machine Learning course on Coursera. This week, we have been assigned a bit of a difficult problem of implementing backpropagation. But before I could even get started on that, I'm facing some very silly and proportionally frustrating issues.

I am using writing some simple code, like adding a column of 1s to a matrix by concatening. The matrix is X of dimensions 5000 x 400. Therefore, after this command it should be come 5000 x 401. But that's simply not happening. The code I'm using is this.

X = [ones(m,1) X];

I have tried the exact same line (copy-pasted) on the command line and it works perfectly, it gives X as 5000 X 401. I have gone through some other forum questions: some had problems where the name of the function was already a MatLab keyword. But that doesn't seem to be the problem here.

Here's the entire function code. Please go through it.

function [J, grad] = nnCostFunction(nn_params, ...
                                   input_layer_size, ...
                                   hidden_layer_size, ...
                                   num_labels, ...
                                   X, y, lambda)
%NNCOSTFUNCTION Implements the neural network cost function for a two layer
%neural network which performs classification
%   [J grad] = NNCOSTFUNCTON(nn_params, hidden_layer_size, num_labels, ...
%   X, y, lambda) computes the cost and gradient of the neural network. The
%   parameters for the neural network are "unrolled" into the vector
%   nn_params and need to be converted back into the weight matrices. 
% 
%   The returned parameter grad should be a "unrolled" vector of the
%   partial derivatives of the neural network.
%

% Reshape nn_params back into the parameters Theta1 and Theta2, the weight matrices
% for our 2 layer neural network
Theta1 = reshape(nn_params(1:hidden_layer_size * (input_layer_size + 1)), ...
                 hidden_layer_size, (input_layer_size + 1));

Theta2 = reshape(nn_params((1 + (hidden_layer_size * (input_layer_size + 1))):end), ...
                 num_labels, (hidden_layer_size + 1));

% Setup some useful variables
m = size(X, 1);
         
% You need to return the following variables correctly 
J = 0;
X = [ones(m,1) X];
Theta1_grad = zeros(size(Theta1));
Theta2_grad = zeros(size(Theta2));

% ====================== YOUR CODE HERE ======================
% Instructions: You should complete the code by working through the
%               following parts.
%
% Part 1: Feedforward the neural network and return the cost in the
%         variable J. After implementing Part 1, you can verify that your
%         cost function computation is correct by verifying the cost
%         computed in ex4.m
%
% Part 2: Implement the backpropagation algorithm to compute the gradients
%         Theta1_grad and Theta2_grad. You should return the partial derivatives of
%         the cost function with respect to Theta1 and Theta2 in Theta1_grad and
%         Theta2_grad, respectively. After implementing Part 2, you can check
%         that your implementation is correct by running checkNNGradients
%
%         Note: The vector y passed into the function is a vector of labels
%               containing values from 1..K. You need to map this vector into a 
%               binary vector of 1's and 0's to be used with the neural network
%               cost function.
%
%         Hint: We recommend implementing backpropagation using a for-loop
%               over the training examples if you are implementing it for the 
%               first time.
%
% Part 3: Implement regularization with the cost function and gradients.
%
%         Hint: You can implement this around the code for
%               backpropagation. That is, you can compute the gradients for
%               the regularization separately and then add them to Theta1_grad
%               and Theta2_grad from Part 2.
%

% adding bias term to each row in X


% eye_matrix = eye(num_labels);
% y_matrix = eye_matrix(y,:);

% -------------------------------------------------------------

% =========================================================================

% Unroll gradients
grad = [Theta1_grad(:) ; Theta2_grad(:)];

end

Also, I tried the code with the y_matrix and that wasn't working either. By the way, interestingly, the J = 0 code is working fine (I tried to change the value of J to see if it would reflected in the workspace and it was), meaning that main .mlx file is able to run this function. But these specific lines of code that is, of X don't work.

The code in the .mlx file used to run this is as follows:

load('ex4data1.mat');
m = size(X, 1);

% Randomly select 100 data points to display
sel = randperm(size(X, 1));
sel = sel(1:100);

displayData(X(sel, :));

% Load the weights into variables Theta1 and Theta2
load('ex4weights.mat');

input_layer_size  = 400;  % 20x20 Input Images of Digits
hidden_layer_size = 25;   % 25 hidden units
num_labels = 10;          % 10 labels, from 1 to 10 (note that we have mapped "0" to label 10)

% Unroll parameters 
nn_params = [Theta1(:) ; Theta2(:)];

% Weight regularization parameter (we set this to 0 here).
lambda = 0;

J = nnCostFunction(nn_params, input_layer_size, hidden_layer_size, num_labels, X, y, lambda);

fprintf('Cost at parameters (loaded from ex4weights): %f', J);

Sorry for the huge comments, perhaps they'd help you understand the objective of the code though that isn't the problem. I'm just not even able to simply execute a simple matrix concatenation.

EDIT::

So I went on a GitHub repo and copy-pasted code to see if that made it work. It did work and did give the right answer but in the workspace it still shows the dimensions of X as 5000 x 400 instead of 5000 x 401. This makes me think that maybe when we come out of the function maybe the values of X and y are reset. I can't know for sure.

[Ameer Hamza]

What is the complete error message?

[Aditya Shukla]

There is no error message. The code syntax seems to be fine but it won't show the right dimensions for X. Plus, if I try to intilize y_matrix, that won't show up in the Workspace either.

[Ameer Hamza]

What are the dimensions of X before and after running this line? Have you tried adding a breakpoint at that line?

[Aditya Shukla]

X is 5000 x 400 originally and does not change after running this line. What do you mean by a breakpoint? From what I gather, a breakpoint is used to do error-testing? Again, I haven't received an error.

[Walter Roberson]

Please attach ex4data1.mat

[Aditya Shukla]

It's larger than 5MB, I'm not able to attach it. But X was 5000 x 400 and y was 5000 x 1.

[Ameer Hamza]

No, breakpoints are not just for syntax errors. They can also be used for logical errors. See here: https://www.mathworks.com/help/matlab/matlab_prog/set-breakpoints.html. Add a breakpoint at this line, the execution will halt, and you can see the value of variables inside the function workspace. From there, you can see what the issue is.

Also, what are the dimensions of nn_params?

[Aditya Shukla]

I figured it out!! I added a breakpoint there and a disp statement to print the size of X. It was indeed 5000 x 401. I think I was right, the value of X gets reset to its original value after the function call.

Thanks everybody for your help!

Answer 1

[John D'Errico]

You change grad inside nnCostFunction. But do you RETURN grad? No. What arguments do you return?

J = nnCostFunction(nn_params, input_layer_size, hidden_layer_size, num_labels, X, y, lambda);

Only J. The updated grad gets dumped in the bit bucket. This is how MATLAB works. Just because your function CAN potentially return that second argument, if you don't put it anywhere, it gets dumped.

[Aditya Shukla]

I do return grad. But I haven't written code to compute it as of yet. It's a backpropagation assignment so I use the backprop algorithm to calculate the partial derivative and then return grad. Do you think that could be the problem? Should I try removing grad from the function definition and call?

[Aditya Shukla]

I figured it out!! I added a breakpoint there and a disp statement to print the size of X. It was indeed 5000 x 401. I think I was right, the value of X gets reset to its original value after the function call.

Thank you for your help!