If you create a custom deep learning layer, then you can use
the checkLayer
function
to check that the layer is valid. The function checks layers for validity, GPU compatibility,
and correctly defined gradients. To check that a layer is valid, run the following
command:
checkLayer(layer,validInputSize,'ObservationDimension',dim)
layer
is an instance of the layer, validInputSize
is a
vector or cell array specifying the valid input sizes to the layer, and dim
specifies the dimension of the observations in the layer input data. For large input sizes, the gradient checks take longer to run. To speed up the tests, specify a smaller valid input size.Check the validity of the example custom layer preluLayer
.
Define a custom PReLU layer. To create this layer, save the file preluLayer.m
in the current folder.
Create an instance of the layer and check that it is valid using checkLayer
. Set the valid input size to the typical size of a single observation input to the layer. For a single input, the layer expects observations of size hbywbyc, where h, w, and c are the height, width, and number of channels of the previous layer output, respectively.
Specify validInputSize
as the typical size of an input array.
layer = preluLayer(20,'prelu');
validInputSize = [5 5 20];
checkLayer(layer,validInputSize)
Skipping multiobservation tests. To enable tests with multiple observations, specify the 'ObservationDimension' option in checkLayer. For 2D image data, set 'ObservationDimension' to 4. For 3D image data, set 'ObservationDimension' to 5. For sequence data, set 'ObservationDimension' to 2. Skipping GPU tests. No compatible GPU device found. Running nnet.checklayer.TestCase .......... ... Done nnet.checklayer.TestCase __________ Test Summary: 13 Passed, 0 Failed, 0 Incomplete, 11 Skipped. Time elapsed: 3.4818 seconds.
The results show the number of passed, failed, and skipped tests. If you do not specify the 'ObservationsDimension'
option, or do not have a GPU, then the function skips the corresponding tests.
Check Multiple Observations
For multiobservation input, the layer expects an array of observations of size hbywbycbyN, where h, w, and c are the height, width, and number of channels, respectively, and N is the number of observations.
To check the layer validity for multiple observations, specify the typical size of an observation and set 'ObservationDimension'
to 4.
layer = preluLayer(20,'prelu'); validInputSize = [5 5 20]; checkLayer(layer,validInputSize,'ObservationDimension',4)
Skipping GPU tests. No compatible GPU device found. Running nnet.checklayer.TestCase .......... ........ Done nnet.checklayer.TestCase __________ Test Summary: 18 Passed, 0 Failed, 0 Incomplete, 6 Skipped. Time elapsed: 3.1679 seconds.
In this case, the function does not detect any issues with the layer.
The checkLayer
function checks the validity of a custom layer by performing a series of tests.
The checkLayer
function uses these tests to check the validity of custom
intermediate layers (layers of type nnet.layer.Layer
).
Test  Description 

methodSignaturesAreCorrect  The syntaxes of the layer functions are correctly defined. 
predictDoesNotError  predict does not error. 
forwardDoesNotError 

forwardPredictAreConsistentInSize 

backwardDoesNotError  backward does not error. 
backwardIsConsistentInSize  The outputs of

predictIsConsistentInType  The outputs of 
forwardIsConsistentInType  The outputs of 
backwardIsConsistentInType  The outputs of 
gradientsAreNumericallyCorrect  The gradients computed in backward are consistent
with the numerical gradients. 
The tests predictIsConsistentInType
, forwardIsConsistentInType
, and backwardIsConsistentInType
also check for GPU compatibility. To execute the layer functions on a GPU, the functions must support inputs and outputs of type gpuArray
with the underlying data type single
.
If you have not implemented forward
, then checkLayer
does not run the forwardDoesNotError
, forwardPredictAreConsistentInSize
, and forwardIsConsistentInType
tests.
The checkLayer
function uses these tests to check the
validity of custom output layers (layers of type
nnet.layer.ClassificationLayer
or
nnet.layer.RegressionLayer
).
Test  Description 

forwardLossDoesNotError  forwardLoss does not error. 
backwardLossDoesNotError  backwardLoss does not error. 
forwardLossIsScalar  The output of forwardLoss is scalar. 
backwardLossIsConsistentInSize  The output of backwardLoss is consistent in size: dLdY is the same size as the predictions Y . 
forwardLossIsConsistentInType  The output of 
backwardLossIsConsistentInType  The output of 
gradientsAreNumericallyCorrect  The gradients computed in backwardLoss are numerically correct. 
The forwardLossIsConsistentInType
and backwardLossIsConsistentInType
tests also check for GPU compatibility. To execute the layer functions on a GPU, the functions must support inputs and outputs of type gpuArray
with the underlying data type single
.
To check the layer validity, the checkLayer
function generates data depending on the type of layer:
Layer Type  Description of Generated Data 

Intermediate  Values in the range [1,1] 
Regression output  Predictions and targets with values in the range [1,1] 
Classification output  Predictions with values in the range [0,1]. If
you specify the If you do
not specify the 
To check for multiple observations, specify the observation dimension using the 'ObservationDimension'
namevalue pair. If you specify the observation dimension, then the checkLayer
function checks that the layer functions are valid using generated data with minibatches of size 1 and 2. If you do not specify this namevalue pair, then the function skips the tests that check that the layer functions are valid for multiple observations.
If a test fails when you use checkLayer
,
then the function provides a test diagnostic and a framework diagnostic. The test
diagnostic highlights any issues found with the layer. The framework diagnostic provides
more detailed information.
The test methodSignaturesAreCorrect
checks that the layer
functions have correctly defined syntaxes.
Test Diagnostic  Description  Possible Solution 

Incorrect number of input arguments for 'predict' in
Layer .  The syntax for predict is not consistent with
the number of layer inputs.  Specify the correct number of input and output
arguments in The syntax for [Z1,…,Zm] = predict(layer,X1,…,Xn) X1,…,Xn are the n layer inputs and
Z1,…,Zm are the m layer outputs. The values
n and m must correspond to the
NumInputs and NumOutputs properties of the
layer.TipIf the number of inputs to 
Incorrect number of output arguments for 'predict' in
Layer  The syntax for predict is not consistent with
the number of layer outputs.  
Incorrect number of input arguments for 'forward' in
Layer  The syntax for forward is not consistent with
the number of layer inputs.  Specify the correct number of input and output
arguments in The syntax for [Z1,…,Zm,memory] = forward(layer,X1,…,Xn) X1,…,Xn are the n layer inputs, Z1,…,Zm are the m layer outputs, and memory is the memory of the layer.TipIf the number of inputs to 
Incorrect number of output arguments for 'forward' in
Layer  The syntax for forward is not consistent with
the number of layer outputs.  
Incorrect number of input arguments for 'backward' in
Layer  The syntax for backward is not consistent with
the number of layer inputs and outputs.  Specify the correct number of input and output
arguments in The syntax for [dLdX1,…,dLdXn,dLdW1,…,dLdWk] = backward(layer,X1,…,Xn,Z1,…,Zm,dLdZ1,…,dLdZm,memory) X1,…,Xn are the n layer inputs,
Z1,…,Zm are the m outputs of
forward , dLdZ1,…,dLdZm are the gradients backward
propagated from the next layer, and memory is the memory output of
forward . For the outputs, dLdX1,…,dLdXn are the
derivatives of the loss with respect to the layer inputs and
dLdW1,…,dLdWk are the derivatives of the loss with respect to the
k learnable parameters. To reduce memory usage by preventing unused
variables being saved between the forward and backward pass, replace the corresponding input
arguments with ~ .TipIf the number of inputs to If the number of outputs can vary, then use 
Incorrect number of output arguments for 'backward' in
Layer  The syntax for backward is not consistent with
the number of layer outputs. 
For layers with multiple inputs or outputs, you must set the values of the layer
properties NumInputs
(or alternatively,
InputNames
) and NumOutputs
(or
alternatively, OutputNames
) in the layer constructor function,
respectively.
The checkLayer
function checks that the layer functions are
valid for single and multiple observations. To check for multiple observations, specify the observation dimension using the 'ObservationDimension'
namevalue pair. If you specify the observation dimension, then the checkLayer
function checks that the layer functions are valid using generated data with minibatches of size 1 and 2. If you do not specify this namevalue pair, then the function skips the tests that check that the layer functions are valid for multiple observations.
Test Diagnostic  Description  Possible Solution 

Skipping multiobservation tests. To enable checks with
multiple observations, specify the 'ObservationDimension'
parameter in checkLayer .  If you do not specify the
'ObservationDimension' parameter in
checkLayer , then the function skips the
tests that check data with multiple observations.  Use the command
For more information, see Layer Input Sizes. 
These tests check that the layers do not error when passed input data of valid size.
Intermediate Layers. The tests predictDoesNotError
,
forwardDoesNotError
, and
backwardDoesNotError
check that the layer functions do
not error when passed inputs of valid size. If you specify an observation
dimension, then the function checks the layer for both a single observation and
multiple observations.
Test Diagnostic  Description  Possible Solution 

The function 'predict' threw an
error:  The predict function errors when
passed data of size
validInputSize .  Address the error described in the
Framework Diagnostic section. 
The function 'forward' threw an
error:  The forward function errors when
passed data of size
validInputSize .  
The function 'backward' threw an
error:  The backward function errors when
passed the output of predict . 
Output Layers. The tests forwardLossDoesNotError
and
backwardLossDoesNotError
check that the layer functions
do not error when passed inputs of valid size. If you specify an observation
dimension, then the function checks the layer for both a single observation and
multiple observations.
Test Diagnostic  Description  Possible Solution 

The function 'forwardLoss' threw an
error:  The forwardLoss function errors when
passed data of size
validInputSize .  Address the error described in the
Framework Diagnostic section. 
The function 'backwardLoss' threw an
error:  The backwardLoss function errors when
passed data of size
validInputSize . 
These tests check that the layer function outputs are consistent in size.
Intermediate Layers. The test backwardIsConsistentInSize
checks that the
backward
function outputs derivatives of the correct
size.
The syntax for backward
is
[dLdX1,…,dLdXn,dLdW1,…,dLdWk] = backward(layer,X1,…,Xn,Z1,…,Zm,dLdZ1,…,dLdZm,memory)
X1,…,Xn
are the n
layer inputs,
Z1,…,Zm
are the m
outputs of
forward
, dLdZ1,…,dLdZm
are the gradients backward
propagated from the next layer, and memory
is the memory output of
forward
. For the outputs, dLdX1,…,dLdXn
are the
derivatives of the loss with respect to the layer inputs and
dLdW1,…,dLdWk
are the derivatives of the loss with respect to the
k
learnable parameters. To reduce memory usage by preventing unused
variables being saved between the forward and backward pass, replace the corresponding input
arguments with ~
.If the number of inputs to backward
can vary, then use
varargin
instead of the input arguments after
layer
. In this case, varargin
is a cell array
of the inputs, where varargin{i}
corresponds to Xi
for i
=1,…,NumInputs
,
varargin{NumInputs+j}
and
varargin{NumInputs+NumOutputs+j}
correspond to
Zj
and dLdZj
, respectively, for
j
=1,…,NumOutputs
, and
varargin{end}
corresponds to memory
.
If the number of outputs can vary, then use varargout
instead of the
output arguments. In this case, varargout
is a cell array of the
outputs, where varargout{i}
corresponds to dLdXi
for i
=1,…,NumInputs
and
varargout{NumInputs+t}
corresponds to dLdWt
for t
=1,…,k
, where k
is the
number of learnable parameters.
The derivatives dLdX1
, …, dLdXn
must be
the same size as the corresponding layer inputs, and
dLdW1,…,dLdWk
must be the same size as the corresponding
learnable parameters. The sizes must be consistent for input data with single
and multiple observations.
Test Diagnostic  Description  Possible Solution 

Incorrect size of 'dLdX' for
'backward' .  The derivatives of the loss with respect to the layer inputs must be the same size as the corresponding layer input.  Return the derivatives

Incorrect size of the derivative of the loss
with respect to the input 'in1' for
'backward'  
The size of 'Z' returned from 'forward' must be
the same as for 'predict' .  The outputs of predict must be the
same size as the corresponding outputs of
forward .  Return the outputs 
Incorrect size of the derivative of the loss
with respect to 'W' for 'backward' .  The derivatives of the loss with respect to the learnable parameters must be the same size as the corresponding learnable parameters.  Return the derivatives

Output Layers. The tests forwardLossIsScalar
and
backwardLossIsConsistentInSize
check that the outputs of
forwardLoss
and backwardLoss
are of
the correct size.
The syntax for forwardLoss
is loss
= forwardLoss(layer, Y, T)
. The input Y
corresponds to the
predictions made by the network. These predictions are the output of the previous layer. The
input T
corresponds to the training targets. The output
loss
is the loss between Y
and T
according to the specified loss function. The output loss
must be
scalar.
The syntax for backwardLoss
is dLdY
= backwardLoss(layer, Y, T)
. The inputs Y
are the predictions
made by the network and T
are the training targets. The output
dLdY
is the derivative of the loss with respect to the predictions
Y
. The output dLdY
must be the same size as the layer
input Y
.
Test Diagnostic  Description  Possible Solution 

Incorrect size of 'loss' for
'forwardLoss' .  The output loss of
forwardLoss must be a scalar.  Return the output 
Incorrect size of the derivative of loss 'dLdY'
for 'backwardLoss' .  The derivatives of the loss with respect to the layer input must be the same size as the layer input.  Return derivative dLdY with the same
size as the layer input Y . 
These tests check that the layer function outputs are consistent in type and that the layer functions are GPU compatible.
For GPU compatibility, the layer functions must support inputs
and return outputs of type gpuArray
. Any other functions the layer uses
must do the same. Many MATLAB^{®} builtin functions support gpuArray
input arguments. If you call any of these functions with at least one
gpuArray
input, then the function executes on the GPU and returns a
gpuArray
output. For a list of functions that execute on a GPU, see
Run MATLAB Functions on a GPU (Parallel Computing Toolbox). To use a GPU for deep
learning, you must also have a CUDA^{®} enabled NVIDIA^{®} GPU with compute capability 3.0 or higher. For more information on working with GPUs in MATLAB, see GPU Computing in MATLAB (Parallel Computing Toolbox).
Intermediate Layers. The tests predictIsConsistentInType
,
forwardIsConsistentInType
, and
backwardIsConsistentInType
check that the layer functions
output variables of the correct data type. The tests check that the layer
functions return consistent data types when given inputs of the data types
single
, double
, and
gpuArray
with the underlying types
single
or double
.
If you preallocate arrays using functions like
zeros
, then you must ensure that the data types of these arrays are
consistent with the layer function inputs. To create an array of zeros of the same data type of
another array, use the 'like'
option of zeros
. For
example, to initialize an array of zeros of size sz
with the same data type
as the array X
, use Z = zeros(sz,'like',X)
.
Test Diagnostic  Description  Possible Solution 

Incorrect type of 'Z' for
'predict' .  The types of the outputs
Z1,…,Zm of predict
must be consistent with the inputs
X1,…,Xn .  Return the outputs

Incorrect type of output 'out1' for
'predict' .  
Incorrect type of 'Z' for
'forward' .  The types of the outputs
Z1,…,Zm of forward
must be consistent with the inputs
X1,…,Xn .  
Incorrect type of output 'out1' for
'forward' .  
Incorrect type of 'dLdX' for
'backward' .  The types of the derivatives
dLdX1,…,dLdXn of
backward must be consistent with the
inputs X1,…,Xn .  Return the derivatives

Incorrect type of the derivative of the loss
with respect to the input 'in1' for
'backward' .  
Incorrect type of the derivative of loss with
respect to 'W' for 'backward' .  The type of the derivative of the loss of the learnable parameters must be consistent with the corresponding learnable parameters.  For each learnable parameter, return the derivative with the same type as the corresponding learnable parameter. 
Output Layers. The tests forwardLossIsConsistentInType
and
backwardLossIsConsistentInType
check that the layer
functions output variables of the correct data type. The tests check that the
layers return consistent data types when given inputs of the data types
single
, double
, and
gpuArray
with the underlying types
single
or double
.
Test Diagnostic  Description  Possible Solution 

Incorrect type of 'loss' for
'forwardLoss' .  The type of the output loss of
forwardLoss must be consistent with
the input Y .  Return 
Incorrect type of the derivative of loss 'dLdY'
for 'backwardLoss' .  The type of the output dLdY of
backwardLoss must be consistent with
the input Y .  Return 
The test gradientsAreNumericallyCorrect
checks that the
gradients computed by the layer functions are numerically correct.
Intermediate Layers. The test gradientsAreNumericallyCorrect
tests that the
gradients computed in backward
are numerically
correct.
Test Diagnostic  Description  Possible Solution 

The derivative 'dLdX' for 'backward' is
inconsistent with the numerical
gradient .  One or more of the following:
 Check that the derivatives in
If the derivatives are correctly
computed, then in the If the absolute and relative errors are within an acceptable margin of the tolerance, then you can ignore this test diagnostic. 
The derivative of the loss with respect to the
input 'in1' for 'backward' is inconsistent with the
numerical gradient .  
The derivative of loss with respect to 'W' for
'backward' is inconsistent with the numerical
gradient . 
Output Layers. The test gradientsAreNumericallyCorrect
checks that the
gradients computed in backwardLoss
are numerically
correct.
Test Diagnostic  Description  Possible Solution 

The derivative 'dLdY' for 'backwardLoss' is
inconsistent with the numerical
gradient .  One or more of the following:
 Check that the derivatives in
If the derivatives are correctly
computed, then in the If the absolute and relative errors are within an acceptable margin of the tolerance, then you can ignore this test diagnostic. 