train a neural network

I tried to create a new neural network for function approxiamting. I used the neural network designer tool. I generated the code, however it didn't go well.

I tried to use a sequentila input layer -> fc layer, but MATLAB required a unfolding layer in between. later I noticed I need a folding layer after the unfolding layer, but that didn't work as well. the current netwrk has both the above layer, however it doesn't work.

would love to get some help.

the code:

 [X1, X2] = meshgrid(-2:.2:2, -2:.2:2); 
    Y = X1 .* exp(-X1.^2 - X2.^2); 
    % figure();
    % surf(X1, X2, Y)
     
    Ntrain  = 500;
    X_train = (4 * rand(2, Ntrain) - 2).';
    Y_train = X_train(:, 1) .* exp(-X_train(:, 1).^2 - X_train(:, 2).^2);
    % tbl = table(X_train, Y_train);
     
    % define network's layers
    lgraph = layerGraph();
    tempLayers = [
        sequenceInputLayer([1 500 2],"Name","sequence")
        sequenceFoldingLayer("Name","seqfold")
        sequenceUnfoldingLayer("Name","sequnfold")
        fullyConnectedLayer(4,"Name","fc_1")
        tanhLayer("Name","tanh_1")
        fullyConnectedLayer(3,"Name","fc_2")
        tanhLayer("Name","tanh_2")
        fullyConnectedLayer(1,"Name","fc_3")
        regressionLayer("Name","regressionoutput")];
    lgraph = addLayers(lgraph,tempLayers);
    lgraph = connectLayers(lgraph,"seqfold/miniBatchSize","sequnfold/miniBatchSize");
    clear tempLayers;
    plot(lgraph);
    % define network's options
    options = trainingOptions('sgdm', ...
        'LearnRateSchedule','piecewise', ...
        'LearnRateDropFactor',0.2, ...
        'LearnRateDropPeriod',5, ...
        'MaxEpochs',20, ...
        'MiniBatchSize',50, ...
        'Plots','training-progress');
     
    % analyzeNetwork(layers)
    net = trainNetwork(X_train, Y_train, layers, options);

the error:

Error using trainNetwork (line 165)
    Invalid network.
    
    Error in main (line 53)
    net = trainNetwork(X_train, Y_train, layers, options);
    
    Caused by:
        Layer 'fc_1': Invalid network. Layer does not support sequence input. Try using a sequence folding layer before the layer.

Thanks!

[Anouar Yatribi]

Hi Nitai,

I have the same issue when trying to implement a custom layer after a sequence input layer. I haven't found any solution to this problem. Can you please tell us if you solved it ?

Thanks in advance.