analyzeNetwork

Analyze deep learning network architecture

Syntax

analyzeNetwork(net)

analyzeNetwork(net,X1,...,Xn)

info = analyzeNetwork(___)

___ = analyzeNetwork(___,Plots=plotName)

Description

Use analyzeNetwork to visualize and understand the architecture of a network, check that you have defined the architecture correctly, and detect problems before training. Problems that analyzeNetwork detects include incorrectly sized layer inputs, an incorrect number of layer inputs, and invalid neural network structures.

Tip

To interactively build and visualize deep learning neural networks, use the Deep Network Designer app. For more information, see Get Started with Deep Network Designer.

analyzeNetwork(net) analyzes and detects errors and issues in the specified network or layer array. The function displays an interactive visualization of the network architecture and provides detailed information. The information includes the layer types, the sizes and formats of the layer learnable parameters, states, and activations, and the total number of learnable parameters.

If the network contains ProjectedLayer objects, then the function additionally displays information about the percentage of learnables removed by projection.

If the network is a taylorPrunableNetwork object, then the function additionally displays information about the proportion of learnables removed by pruning and the number of pruned filters.

Each activation dimension has one of these labels: "S" (spatial), "C" (channel), "B" (batch), "T" (time or sequence), or "U" (unspecified).

analyzeNetwork(net,X1,...,Xn) analyzes the neural network using the specified example network inputs. The software propagates the example inputs through the network to determine the size and format of layer activations, the size and number of learnable and state parameters, and the total number of learnables. Use this syntax to analyze neural networks that have one or more inputs that are not connected to an input layer.

info = analyzeNetwork(___) also returns a NetworkAnalysis object. Use this object to programmatically access the analysis results. For example, you can access the total number of learnables, layer information, and network issues.

___ = analyzeNetwork(___,Plots=plotName) also specifies which plots to display during the network analysis. To analyze the network programmatically, without opening the analysis plot, set the Plots option to "none" .

Examples

Analyze Trained Network

Load a pretrained GoogLeNet convolutional neural network.

net = imagePretrainedNetwork("googlenet");

Analyze the network. analyzeNetwork displays an interactive plot of the network architecture and a table containing information about the network layers.

info = analyzeNetwork(net);

Investigate the network architecture using the plot to the left. Select a layer in the plot. The selected layer is highlighted in the plot and in the layer table.

In the table, view layer information such as layer properties, layer type, and sizes of the layer activations and learnable parameters. The activations of a layer are the outputs of that layer. Each activation dimension has one of the following labels:

• S — Spatial

• C — Channel

• B — Batch observations

• T — Time or sequence

• U — Unspecified

View the dimension labels to understand how data propagates through the network and how the layers modify the size and layout of activations.

Select a deeper layer in the network. Notice that activations in deeper layers are smaller in the spatial dimensions (the first two dimensions) and larger in the channel dimension (the last dimension). Using this structure enables convolutional neural networks to gradually increase the number of extracted image features while decreasing the spatial resolution.

The Deep Learning Network Analyzer shows the total number of learnable parameters in the network, to one decimal place. To see the exact number of learnable parameters, pause on total learnables. To show the number of learnable parameters in each layer, click the arrow in the top-right corner of the layer table and select Number of Learnables. To sort the layer table by column value, hover the mouse over the column heading and click the arrow that appears. For example, you can determine which layer contains the most parameters by sorting the layers by the number of learnable parameters.

Programmatically view the network analysis results.

info

info = 
  NetworkAnalysis with properties:

    TotalLearnables: 6998552
          LayerInfo: [143×7 table]
             Issues: [0×3 table]
          NumErrors: 0
        NumWarnings: 0
       AnalysisDate: 11-Dec-2023 17:48:12

Fix Errors in Network Architecture

Create a simple convolutional network with some skip connections.

net = dlnetwork;

layers = [
    imageInputLayer([32 32 3])
    convolution2dLayer(5,16,Padding="same")
    reluLayer(Name="relu_1")
    convolution2dLayer(3,16,Padding="same",Stride=2)
    reluLayer
    additionLayer(2,Name="add_1")
    convolution2dLayer(3,16,Padding="same",Stride=2)
    reluLayer
    additionLayer(3,Name="add_2")
    fullyConnectedLayer(10)
    softmaxLayer];

net = addLayers(net,layers);

layer = convolution2dLayer(1,16,Stride=2,Name="conv_skip");
net = addLayers(net,layer);
net = connectLayers(net,"relu_1","add_1/in2");
net = connectLayers(net,"add_1","add_2/in2");

Analyze the network architecture using the analyzeNetwork function. The function finds issues with three layers in the network.

analyzeNetwork(net)

Investigate and fix the errors in the network. In this example, the following issues cause the errors:

• The network has inputs that are not connected to an input layer. This is not a requirement for a valid neural network, but for networks with inputs that are not connected to an input layer, the analyzeNetwork function requires example input data so it can propagate it through the network to identify downstream issues. Because the underlying issue for this network is that there are missing connections or misconfigured layers, you can ignore this error.

• The "add_2" layer is configured to have three inputs, but the layer only has two inputs. To fix the error, specify the number of inputs as 2.

• The layer "conv_skip" has an unconnected input. It should be a part of the shortcut connection between the add_1 and add_2 layers. To fix this error, connect "add_1" to "conv_skip" and "conv_skip" to "add_2".

• All the inputs to an addition layer must have the same size, but the "add_1" layer has two inputs with different sizes. Because the "conv_2" layer has a Stride of 2, the layer downsamples the activations by a factor of two in the first two dimensions (the spatial dimensions). To resize the input from the "relu_2" layer so that it has the same size as the input from "relu_1", remove the downsampling by removing the Stride value of the "conv_2" layer.

Apply these modifications to the neural network construction from the beginning of this example and create a new network.

net = dlnetwork;

layers = [
    imageInputLayer([32 32 3])
    convolution2dLayer(5,16,Padding="same")
    reluLayer(Name="relu_1")
    convolution2dLayer(3,16,Padding="same")
    reluLayer
    additionLayer(2,Name="add_1")
    convolution2dLayer(3,16,Padding="same",Stride=2)
    reluLayer
    additionLayer(2,Name="add_2")
    fullyConnectedLayer(10)
    softmaxLayer];

net = addLayers(net,layers);

layer = convolution2dLayer(1,16,Stride=2,Name="conv_skip");
net = addLayers(net,layer);

net = connectLayers(net,"relu_1","add_1/in2");
net = connectLayers(net,"add_1","conv_skip");
net = connectLayers(net,"conv_skip","add_2/in2");

Analyze the new architecture. The new network does not contain any errors and is ready to train.

analyzeNetwork(net)

Analyze Network Using Example Inputs

To analyze a network that has an input that is not connected to an input layer, you can provide example network inputs to the analyzeNetwork function.

Define the network architecture. Construct a network with two branches. The network takes two inputs, with one input per branch. Connect the branches using an addition layer.

numFilters = 24;
inputSize = [64 64 3];

net = dlnetwork;

layersBranch1 = [
    convolution2dLayer(3,6*numFilters,Padding="same",Stride=2)
    groupNormalizationLayer("all-channels")
    reluLayer
    convolution2dLayer(3,numFilters,Padding="same")
    groupNormalizationLayer("channel-wise")
    additionLayer(2,Name="add")
    reluLayer
    fullyConnectedLayer(10)
    softmaxLayer];

layersBranch2 = [
    convolution2dLayer(1,numFilters,Name="conv_branch")
    groupNormalizationLayer("all-channels",Name="gn_branch")];

net = addLayers(net,layersBranch1);
net = addLayers(net,layersBranch2);
net = connectLayers(net,"gn_branch","add/in2");

View the network input names.

net.InputNames

ans = 1×2 cell
    {'conv_1'}    {'conv_branch'}

Create example network inputs of the same size and format as typical inputs for this network using random dlarray objects. For both inputs, use a batch size of 32. Use an input of size 64-by-64 with three channels for the input to the layer "input". Use an input of size 64-by-64 with 18 channels for the input to the layer "conv_branch".

X1 = dlarray(rand([inputSize 32]),"SSCB");
X2 = dlarray(rand([32 32 18 32]),"SSCB");

Analyze the network. Provide the example inputs in the same order as the InputNames property of the dlnetwork. You must provide an example input for all network inputs, including inputs that are connected to an input layer.

analyzeNetwork(net,X1,X2)

Analyze Taylor Prunable Network

Load a trained and pruned taylorPrunableNetwork object.

load("prunedDigitsCustom.mat");

Analyze the network. analyzeNetwork displays an interactive plot of the network architecture and a table containing information about the network layers. The table shows the number of pruned convolutional filters. The table also shows the percentage decrease in the number of learnables for each layer. This includes the three convolutional layers, but also downstream effects in other layers that do not have pruned filters.

info = analyzeNetwork(prunableNet);

Programmatically view the layer information table.

info.LayerInfo

ans=12×9 table
      Name               Type             ActivationSizes    ActivationFormats                     LearnableSizes                     NumLearnables                       StateSizes                       LearnablesReduction    NumPrunedFilters
    _________    _____________________    _______________    _________________    ________________________________________________    _____________    ________________________________________________    ___________________    ________________

    "input"      "Image Input"            {[ 28 28 1 1]}        {["SSCB"]}        {[dictionary (string --> cell) with no entries]}            0        {[dictionary (string --> cell) with no entries]}                0                 0        
    "conv1"      "2-D Convolution"        {[24 24 18 1]}        {["SSCB"]}        {[dictionary (string --> cell) with 2 entries ]}          468        {[dictionary (string --> cell) with no entries]}              0.1                 2        
    "bn1"        "Batch Normalization"    {[24 24 18 1]}        {["SSCB"]}        {[dictionary (string --> cell) with 2 entries ]}           36        {[dictionary (string --> cell) with 2 entries ]}              0.1                 0        
    "relu1"      "ReLU"                   {[24 24 18 1]}        {["SSCB"]}        {[dictionary (string --> cell) with no entries]}            0        {[dictionary (string --> cell) with no entries]}                0                 0        
    "conv2"      "2-D Convolution"        {[24 24 18 1]}        {["SSCB"]}        {[dictionary (string --> cell) with 2 entries ]}         2934        {[dictionary (string --> cell) with no entries]}           0.1895                 2        
    "bn2"        "Batch Normalization"    {[24 24 18 1]}        {["SSCB"]}        {[dictionary (string --> cell) with 2 entries ]}           36        {[dictionary (string --> cell) with 2 entries ]}              0.1                 0        
    "relu2"      "ReLU"                   {[24 24 18 1]}        {["SSCB"]}        {[dictionary (string --> cell) with no entries]}            0        {[dictionary (string --> cell) with no entries]}                0                 0        
    "conv3"      "2-D Convolution"        {[24 24 16 1]}        {["SSCB"]}        {[dictionary (string --> cell) with 2 entries ]}         2608        {[dictionary (string --> cell) with no entries]}          0.27956                 4        
    "bn3"        "Batch Normalization"    {[24 24 16 1]}        {["SSCB"]}        {[dictionary (string --> cell) with 2 entries ]}           32        {[dictionary (string --> cell) with 2 entries ]}              0.2                 0        
    "relu3"      "ReLU"                   {[24 24 16 1]}        {["SSCB"]}        {[dictionary (string --> cell) with no entries]}            0        {[dictionary (string --> cell) with no entries]}                0                 0        
    "fc"         "Fully Connected"        {[      10 1]}        {["CB"  ]}        {[dictionary (string --> cell) with 2 entries ]}        92170        {[dictionary (string --> cell) with no entries]}          0.19998                 0        
    "softmax"    "Softmax"                {[      10 1]}        {["CB"  ]}        {[dictionary (string --> cell) with no entries]}            0        {[dictionary (string --> cell) with no entries]}                0                 0        

Input Arguments

net — Neural network architecture

dlnetwork object | layer array | taylorPrunableNetwork object (since R2024a)

Neural network architecture, specified as a dlnetwork object, a layer array, or a taylorPrunableNetwork object.

For a list of built-in neural network layers, see List of Deep Learning Layers.

Analyzing a taylorPrunableNetwork object requires the Deep Learning Toolbox™ Model Quantization Library support package. This support package is a free add-on that you can download using the Add-On Explorer. Alternatively, see Deep Learning Toolbox Model Quantization Library.

plotName — Plots to display during neural network analysis

"analyzer" (default) | "none"

Plots to display during neural network analysis, specified as one of these values:

• "analyzer" — Display analysis plot.

• "none" — Do not display analysis plot.

If you specify plotName as "none", then to access the analysis results, you must return the output of the analyzeNetwork function.

info = analyzeNetwork(net,Plots="none")

X1,...,Xn — Example network inputs

formatted dlarray objects

Example network inputs, specified as formatted dlarray objects. The software propagates the example inputs through the network to determine the size and format of layer activations, the size and number of learnable and state parameters, and the total number of learnables.

Use example inputs when you want to analyze a network that does not have any input layers or that has inputs that are not connected to an input layer.

The order in which you must specify the example inputs depends on the type of network you are analyzing:

• Layer array — Provide example inputs in the same order that the layers that require inputs appear in the Layer array.

• dlnetwork object — Provide example inputs in the same order as the inputs are listed in the InputNames property of the dlnetwork.

• taylorPrunableNetwork object (since R2024a) — Provide example inputs in the same order as the inputs are listed in the InputNames property of the taylorPrunableNetwork.

If a layer has multiple unconnected inputs, then you must specify the example inputs for that layer in the same order as they appear in the InputNames property of the layer.

You must specify one example input for each input to the network, even if that input is connected to an input layer.

Output Arguments

info — Analysis information

NetworkAnalysis object

Analysis information, returned as a NetworkAnalysis object with these properties:

• TotalLearnables — Total number of network learnables

• LayerInfo — Layer information

• Issues — Network issues

• NumErrors — Number of errors

• NumWarnings — Number of warnings

• AnalysisDate — Time of analysis

Version History

Introduced in R2018a

R2024b: Access compression related network analysis information programmatically

Starting in R2024b, the output of the analyzeNetwork function contains compression-specific information if the input network is a taylorPrunableNetwork object or contains a ProjectedLayer object.

R2024a: Access network analysis information programmatically

Starting in R2024a, you can return the output of the analyzeNetwork function. The function returns the analysis information in a NetworkAnalysis object. Use this object to programmatically access the analysis results. For example, you can access the total learnables, layer information, and network issues.

R2024a: Analyze Taylor prunable neural networks and neural networks with projected layers

Starting in R2024a, you can analyze taylorPrunableNetwork objects and neural networks that contain ProjectedLayer objects.

R2024a: SeriesNetwork, DAGNetwork, and LayerGraph objects are not recommended

Starting in R2024a, the trainNetwork function and SeriesNetwork, DAGNetwork, and LayerGraph objects are not recommended. Use the trainnet function and dlnetwork objects instead. This recommendation means that SeriesNetwork, DAGNetwork, and LayerGraph input to the analyzeNetwork function is not recommended.

For SeriesNetwork, DAGNetwork, and LayerGraph, the analyzeNetwork function analyzes the neural network for trainNetwork workflows. This includes checking for input layers and output layers.

R2024a: analyzeNetwork analyzes for dlnetwork objects by default

Starting in R2024a, the analyzeNetwork analyzes layer arrays for dlnetwork object workflows. In previous versions, the analyzeNetwork function analyzes layer arrays for trainNetwork workflows. To reproduce this behavior, use analyzeNetwork(layers,TargetUsage="trainNetwork").

Starting in R2024a, the trainNetwork function and SeriesNetwork, DAGNetwork, and LayerGraph objects are not recommended. Use the trainnet function and dlnetwork objects instead. This recommendation means that SeriesNetwork, DAGNetwork, and LayerGraph input to the analyzeNetwork function is not recommended.

Starting in R2024a, you can return the output of the analyzeNetwork function. The function returns the analysis information in a NetworkAnalysis object. Use this object to programmatically access the analysis results. For example, you can access the total learnables, layer information, and network issues.