Where in documentation does it say that fitnet([]) gives a linear model?

I have seen Greg Heath's posts such as this one in which fitnet([]) is used to fit a linear model, i.e. with no hidden layers. But I can't find a reference in the fitnet documentation that this is the expected result of [] as an input, and I find this hard to spot in the fitnet.m code. Can someone please point me to a description in the documentation of the expected behavior of fitnet([])? I just want to understand this better.

Answer 1

[Greg Heath]

fitnet(H) yields H nodes in the hidden layer

A linear function needs no hidden layer

I could only think of 2 possibilities

 a. net1 = fitnet(0), view(net1)

 b. net2 = fitnet([]), view(net2)

So, as I did, try them out and see what happens.

Not sure if I checked the documentation.

Since I don't have MATLAB on this computer I can't check.

Also can't remember what happens when you try fitnet(0).

Hope this helps.

Greg

[KAE]

fitnet(0) crashes on R2018b, but the diagram of fitnet([]) does show zero hidden layers as expected. (For learners like me: notice the function shown on the right of the 'Output' box is a line.)

So you can compare it to a network with 1 hidden layer and 10 neurons, view(fitnet(10)) is below. The function shown on the right of the 'Hidden' box is a sigmoid, as a reminder that the network can capture non-linear relationships.

Since it is useful to check whether a neural network is outperforming a linear regression (on my type of problem), if no one can finds fitnet([]) documentation I will put a request in to Mathworks.

[the cyclist]

I guess it seems logical enough to me that fitnet(0) would crash, as that specifies one hidden layer of size zero (which is not sensible).

[KAE]

I have contacted Mathworks to confirm that fitnet([]) creates a linear model as we think it does, and will report back.

[Greg Heath]

You don't really need to do that. A demo with

 a. two random positive slope examples

and

 b. two random negative slope examples

should convince even the hardest knuckle-head.

Greg

[KAE]

Here is some code to show that the output fitnet([]) looks like a linear model, for other learners.

slopeList = [+2 +7 -3 -5]; % Slope values that we will use in different cases
offsetList = [2 -1 5 -3]; % Offset values
x = rand(1, 1000); % Independant variable
yNoise = rand(1, 1000) - 0.5; % Add some zero-mean noise
net = fitnet([]); % Fits a linear model
for iCase = 1:length(slopeList)
  % Target is the line plus some noise
  t = x*slopeList(iCase) + offsetList(iCase) + yNoise; 
  net = configure(net, x, t);
  % Train the network on x and t, and return the prediction y
  [net, tr, y] = train(net,x,t); 

    % Try to undo the automatic normalization to get the 
    % unscaled weight and bias (doesn't work right)
    weightBiasUnscaled = ...
        mapminmax('reverse', [net.IW{1} net.b{1}],...
        net.outputs{1}.processSettings{1});
    weightUnscaled = weightBiasUnscaled(1);
    biasUnscaled = weightBiasUnscaled(2); 

    % Show results
    figure;
    h1 = plot(x, t, 'g.'); % Data
    hold on;
    % Prediction from net
    h2 = plot(x, y, '.', 'color', [1 1 1]*0.5, 'markersize', 10); 
    x0 = get(gca, 'xlim');
    h3 = plot(x0, x0*slopeList(iCase) + offsetList(iCase), 'r'); % Real relation
    box off

    % Explain what was calculated
    modelStr = ['Model: y'' = ' sprintf('%.2f', weightUnscaled) ...
      '''*x + ' sprintf('%.1f', biasUnscaled)];
    realStr = ['Real Relation: y = ' num2str(slopeList(iCase)) ...
      '*x + ' num2str(offsetList(iCase))];
    legend('Noisy data', modelStr, realStr);
    title('Linear Model Check');
end

[KAE]

As a side note, I am not sure how to reverse the scaling to regain the unscaled weight and bias values for the linear model y=mx+b. Here is what I tried,

    weightBiasUnscaled = ...
        mapminmax('reverse', [net.IW{1} net.b{1}],...
        net.outputs{1}.processSettings{1});
    % Unscaled weight
    weightUnscaled = weightBiasUnscaled(1);
    % Unscaled bias
    biasUnscaled = weightBiasUnscaled(2);

However this gives a slope (weight) of -8.07 instead of the expected value of m=-5, and a bias (offset) of -5.5 not the expected value of b=-3 in y=mx+b. What am I doing wrong?

[KAE]

MathWorks has confirmed that fitnet([]) produces a linear model like linear regression. They pointed me to this helpful post for background. I have asked them to add info on fitnet([]) to the documentation. Thanks all for your help!

Answer 2

[the cyclist]

In the Input Arguments section of the documentation, where it discusses the hiddenSizes input, there is the sentence, "The length of the vector determines the number of hidden layers in the network."

Since the length of the empty vector is zero:

>> length([])

ans =

     0

one can infer, I suppose, that there will be zero hidden layers. This could certainly be made more explicit, though. (I don't think I would have thought to try, had it not been for your question.)

I don't have access to the code, but I'm guessing that armed with the above knowledge, you can find where MATLAB determines the number of hidden layers from the length of that input.