Maxout activation function for CNN model

I want to implement maxout activation function in AlexNet architecture instead of ReLu activation function. But after lot of searching i am unable to find any pre-defined function or layer in matlab for maxout function just like ReLu layer.

Do I need to create custom layer for implementing maxout function in AlexNet.

If yes, please suggest me how can i create that custom layer for maxout function. Any suggestion will be greatly appreciated.

Thanx alot in advance....

Answer 1

[Darshak]

Hello @priyanka jindal,

I encountered a similar requirement when experimenting with alternative activation functions in MATLAB.

To integrate “Maxout” in place of “ReLU”, you'll need to implement it using a custom layer.

The following is a minimal working example of the custom “Maxout” layer:

classdef MaxoutLayer < nnet.layer.Layer
    properties
        NumPieces
    end
    
    methods
        function layer = MaxoutLayer(numPieces, name)
            layer.Name = name;
            layer.Description = ['Maxout with ', num2str(numPieces), ' pieces'];
            layer.NumPieces = numPieces;
        end
        
        function Z = predict(layer, X)
            inputSize = size(X);
            H = inputSize(1);
            W = inputSize(2);
            N = inputSize(4);
            C = inputSize(3) / layer.NumPieces;
            X = reshape(X, H, W, C, layer.NumPieces, N);
            Z = max(X, [], 4);
        end
    end
end

To substitute a "ReLU" in AlexNet, you can use the "replaceLayer" function within a "layerGraph". This allows selective modification of the network while retaining the pretrained structure.

Make sure that the convolutional layer feeding into the “Maxout” layer produces a number of channels divisible by "NumPieces", as required by the reshape logic.

These are some documentation links which can be useful:

• Reshape - https://www.mathworks.com/help/matlab/ref/reshape.html
• Replace Layer - https://www.mathworks.com/help/deeplearning/ref/nnet.cnn.layergraph.replacelayer.html
• Layer Graph - https://www.mathworks.com/help/deeplearning/ref/nnet.cnn.layergraph.html
• Custom Deep Learning layers - https://www.mathworks.com/help/deeplearning/ug/define-custom-deep-learning-layers.html