idNeuralNetwork
Multilayer neural network mapping function for nonlinear ARX models and Hammerstein-Wiener models (requires Statistics and Machine Learning Toolbox or Deep Learning Toolbox)
Since R2023b
Description
An idNeuralNetwork object creates a neural network function and
is a nonlinear mapping object for estimating nonlinear ARX models and Hammerstein-Wiener
models. This mapping object lets you create neural networks using the regression networks of
Statistics and Machine Learning Toolbox™ and the deep and shallow networks of Deep Learning Toolbox™.
Mathematically, idNeuralNetwork is a function that maps
m inputs X(t) =
[x(t1),x2(t),…,xm(t)]T
to a single scalar output
y(t) using the following relationship:
Here:
X(t) is an m-by-1 vector of inputs, or regressors.
y0 is the output offset, a scalar.
P and Q are m-by-p and m-by-q projection matrices, respectively.
L is a p-by-1 vector of weights.
S(.) represents a neural network object of one of the following types:
RegressionNeuralNetwork(Statistics and Machine Learning Toolbox) object — Network object created usingfitrnet(Statistics and Machine Learning Toolbox)dlnetwork(Deep Learning Toolbox) object — Deep learning network objectnetwork(Deep Learning Toolbox) object — Shallow network object created using a command such asfeedforwardnet(Deep Learning Toolbox)
Additionally, a cascade-correlation neural network is a network type that is implemented with deep networks. It allows you to create networks without specifying the network sizes.
See Examples for more information.
Use idNeuralNetwork as the output value, or, for multiple-output systems,
one of the output values in the OutputFcn property of an idnlarx model or the InputNonlinearity and
OutputNonlinearity properties of an idnlhw object. For example, specify idNeuralNetwork when you
estimate an idnlarx model with the following
command.
sys = nlarx(data,regressors,idNeuralNetwork)
nlarx estimates the model, it essentially estimates the parameters of
the idNeuralNetwork function.You can use a similar approach when you specify input or output linearities using the
nlhw command. For example, specify idNeuralNetwork as both the
input and output nonlinearities with the following command.
sys = nlhw(data,orders,idNeuralNetwork,idNeuralNetwork)
Creation
Syntax
Description
Create Regression Network or Deep Learning Network
NW = idNeuralNetworkidNeuralNetwork object NW that uses a single
hidden layer of ten rectified linear unit (ReLU) activations.
The specific type of network that NW represents depends on the
toolboxes you have access to.
If you have access to Statistics and Machine Learning Toolbox, then
idNeuralNetworkusesfitrnet(Statistics and Machine Learning Toolbox) to create aRegressionNeuralNetwork(Statistics and Machine Learning Toolbox)-based map.If Statistics and Machine Learning Toolbox is not available but you have access to Deep Learning Toolbox, then
idNeuralNetworkusesdlnetwork(Deep Learning Toolbox) to create a deep learning network map.
For idnlhw models, the number of inputs to the network is 1. For
idnlarx models, the number of inputs is unknown, as this number is
determined during estimation. NW also uses a parallel linear
function and an offset element.
For multiple-output nonlinear ARX or Hammerstein-Wiener models, create a separate
idNeuralNetwork object for each output. Each element of the
output function must represent a single-output network object.
NW = idNeuralNetwork(LayerSizes)LayerSizes as the number of layers, if
LayerSizes is a row vector of positive integers. Each
ith element in LayerSizes specifies the number
of activations in the corresponding ith
layer.
NW = idNeuralNetwork(LayerSizes,Activations)Activations specification and the available toolboxes determines
which type of neural network NW uses.
NW = idNeuralNetwork(LayerSizes,Activations,UseLinearFcn)NW uses a linear function as a
subcomponent.
NW = idNeuralNetwork(LayerSizes,Activations,UseLinearFcn,UseOffset)NW uses an offset term.
NW = idNeuralNetwork(___,Name=Value)
Create Cascade-Correlation Neural Network
Creating cascade-correlation neural networks requires Deep Learning Toolbox software. You can use these cascade-correlation neural networks for estimating nonlinear ARX models but not Hammerstein-Wiener models.
NW = idNeuralNetwork("cascade-correlation")
NW = idNeuralNetwork("cascade-correlation",Activations)
NW = idNeuralNetwork("cascade-correlation",Activations,UseLinearFcn)NW uses a linear function as a
subcomponent.
NW = idNeuralNetwork("cascade-correlation",Activations,UseLinearFcn,UseOffset)NW uses an offset term.
NW = idNeuralNetwork(___,Name=Value)
Use Existing Shallow Neural Network
NW = idNeuralNetwork(shallownet)NW using the network (Deep Learning Toolbox) object shallownet.
shallownet is typically the output of feedforwardnet (Deep Learning Toolbox), cascadeforwardnet (Deep Learning Toolbox), or linearlayer (Deep Learning Toolbox).
NW = idNeuralNetwork(shallownet,[],UseLinearFcn)NW uses a linear function as a
subcomponent.
NW = idNeuralNetwork(shallownet,[],UseLinearFcn,UseOffset)NW uses an offset term.
Input Arguments
Name-Value Arguments
Properties
Examples
Create an idNeuralNetwork object with default properties.
NW = idNeuralNetwork
Constructing a RegressionNeuralNetwork object from the Statistics and Machine Learning Toolbox... If you want to use a deep network representation, specify NetworkType="dlnetwork" in the idNeuralNetwork constructor.
NW =
Multi-Layer Neural Network
Nonlinear Function: Uninitialized regression neural network
Contains 1 hidden layers using "relu" activations.
(uses Statistics and Machine Learning Toolbox)
Linear Function: uninitialized
Output Offset: uninitialized
Network: 'Regression neural network parameters'
LinearFcn: 'Linear function parameters'
Offset: 'Offset parameters'
EstimationOptions: [1×1 struct]
NW is a regression neural network with a single layer of relu activations.
Specify a network that uses two hidden layers of sizes 5 and 3 respectively. Specify that both layers use tanh activations.
NW = idNeuralNetwork([5 3],"tanh");Constructing a RegressionNeuralNetwork object from the Statistics and Machine Learning Toolbox... If you want to use a deep network representation, specify NetworkType="dlnetwork" in the idNeuralNetwork constructor.
disp(NW.Network)
Regression neural network parameters
Parameters: 'Learnables and hyperparameters'
Inputs: {1×0 cell}
Outputs: {1×0 cell}
This example assumes that you have access to Statistics and Machine Learning Toolbox, but will also run with Deep Learning Toolbox. If you have access to both toolboxes, then NW is a regression network. If you have access to only Deep Learning Toolbox, then NW is a deep network.
Create a network that contains three hidden layers. The first layer uses 10 relu activations, the second layer uses 5 tanh activations, and the third layer uses swish activations.
NW = idNeuralNetwork([10 5 2],["relu", "tanh", "swish"])
NW =
Multi-Layer Neural Network
Nonlinear Function: Deep learning network
Contains 3 hidden layers using "relu", "tanh", "swish" activations.
(uses Deep Learning Toolbox)
Linear Function: uninitialized
Output Offset: uninitialized
Network: 'Deep learning network parameters'
LinearFcn: 'Linear function parameters'
Offset: 'Offset parameters'
EstimationOptions: [1×1 struct]
The swish network requires Deep Learning Toolbox. Therefore, NW is a deep network whether or not you also have Statistics and Machine Learning Toolbox.
Create an idNeuralNetwork object that has no linear function or offset.
UseLinear = false;
UseOffset = false;
NW = idNeuralNetwork(5,"relu",UseLinear,UseOffset);Constructing a RegressionNeuralNetwork object from the Statistics and Machine Learning Toolbox... If you want to use a deep network representation, specify NetworkType="dlnetwork" in the idNeuralNetwork constructor.
disp(NW.Linear)
Linear Function: not in use
Value: [1×0 double]
Free: [1×0 logical]
Use: 0
Inputs: {1×0 cell}
Outputs: {1×0 cell}
InputProjection: []
disp(NW.Offset)
Output Offset: not in use
Use: 0
Value: NaN
Free: 1
NW does not use the linear function or offset.
Create a network function with default settings, but enforce that the function be based on the deep network architecture.
NW = idNeuralNetwork(NetworkType="dlnetwork")NW =
Multi-Layer Neural Network
Nonlinear Function: Deep learning network
Contains 1 hidden layers using "relu" activations.
(uses Deep Learning Toolbox)
Linear Function: uninitialized
Output Offset: uninitialized
Network: 'Deep learning network parameters'
LinearFcn: 'Linear function parameters'
Offset: 'Offset parameters'
EstimationOptions: [1×1 struct]
The network specification overrides the default selection of a regression network.
Create a cascade-correlation neural network using idNeuralNetwork. Specify the activation function to be "tanh" for all layers.
NW = idNeuralNetwork("cascade-correlation","tanh")
NW =
Multi-Layer Neural Network
Nonlinear Function: Deep learning network
Cascade Correlation network using "tanh" activations.
(uses Deep Learning Toolbox)
Linear Function: uninitialized
Output Offset: uninitialized
Network: 'Deep learning network parameters'
LinearFcn: 'Linear function parameters'
Offset: 'Offset parameters'
EstimationOptions: [1×1 struct]
disp(NW.EstimationOptions.MaxNumActLayers)
20
Change the maximum number of activation layers in the network to 10.
NW.EstimationOptions.MaxNumActLayers = 10;
Construct an idNeuralNetwork object that uses a deep learning shallow network.
Create a feedforward (shallow) network that uses three hidden layers with four, six, and one neurons, respectively.
snet = feedforwardnet([4 6 1]);
Specify the transfer functions for the hidden layers. The output layer uses the default transfer function 'purelin'.
snet.layers{1}.transferFcn = 'logsig';
snet.layers{2}.transferFcn = 'radbas';
snet.layers{3}.transferFcn = 'purelin';Incorporate snet into the idNeuralNetwork object NW.
NW = idNeuralNetwork(snet)
NW =
Multi-Layer Neural Network
Nonlinear Function: Uninitialized shallow network
(uses Deep Learning Toolbox)
Linear Function: uninitialized
Output Offset: uninitialized
Network: 'Shallow network parameters'
LinearFcn: 'Linear function parameters'
Offset: 'Offset parameters'
EstimationOptions: [1×1 struct]
Identify a nonlinear ARX model that uses a regression neural network to describe the regressor-to-output mapping.
Load the data, which consists of the column vectors u and y. Convert the data into a timetable tt with a sample time of 0.8 minutes.
load twotankdata u y tt = timetable(y,u,TimeStep=minutes(0.8));
Split tt into estimation (training) and validation data sets tte and ttv.
tte = tt(1:1000,:); ttv = tt(1001:2000,:);
Specify estimation and search options.
opt = nlarxOptions(Focus="simulation",Display="on",SearchMethod="fmincon"); opt.SearchOptions.MaxIterations = 10;
Create a regression network NW that uses two hidden layers with five activations each. Use sigmoid for the activations in the first layer and tanh for activations in the second layer.
NW = idNeuralNetwork([5 5],["sigmoid","tanh"]);
Constructing a RegressionNeuralNetwork object from Statistics and Machine Learning Toolbox... If you want to use a deep network representation, specify NetworkType="dlnetwork" in the idNeuralNetwork constructor.
Estimate a nonlinear ARX model that uses NW as the output function and uses y as the output variable.
sys = nlarx(tte,[2 2 1],NW,opt,OutputName="y");Using the validation data, compare the measured output with the model output.
compare(ttv,sys)
The nonlinear model shows a good fit to the estimation data.
Identify a nonlinear ARX model that uses a cascade-forward shallow network.
Load the data, which consists of the input and output arrays u and y, respectively. Convert the data into the timetable tt with a sample time of 0.8 minutes.
load twotankdata
tte = timetable(u,y,TimeStep=minutes(0.8));Create a cascade-forward shallow network with a single hidden layer.
cnet = cascadeforwardnet(20);
Construct an idNeuralNetwork object NW that incorporates cnet and excludes the linear and offset elements.
NW = idNeuralNetwork(cnet,[],false,false);
Specify estimation and search options.
opt = nlarxOptions(SearchMethod="gna");
opt.SearchOptions.MaxIterations = 2;Estimate the nlarx model sys and compare the model output with the measured data.
sys = nlarx(tte,[2 2 1],NW,opt);
compare(tte,sys)
Algorithms
The learnable parameters of the idNeuralNetwork function are determined
during estimation of the nonlinear ARX and Hammerstein-Wiener models, using nlarx and nlhw commands, respectively.
The software initializes these parameters using the following steps:
Determine the linear function coefficients L and the offset y0, if in use and free, by performing a least-squares fit to the data.
Initialize the learnable parameters of the network function by fitting the residues of the linear and offset terms from step 1. The initialization scheme depends upon the type of the underlying network:
For
RegressionNeuralNetwork(Statistics and Machine Learning Toolbox) networks, usefitrnet(Statistics and Machine Learning Toolbox).For
dlnetwork(Deep Learning Toolbox) networks, perform initialization by training the network using the specified solver inNW.EstimationOptions. For cascade-correlation neural networks, perform initialization by training a network using the options specified inNW.EstimationOptionsandnlarxOptions.For
network(Deep Learning Toolbox) networks, perform initialization by training the network using the specified solver inNW.EstimationOptions.
After initialization, the software updates the parameters using a nonlinear least-squares
optimization solver (see SearchMethod in nlarxOptions
and SearchOptions in nlhwOptions) to minimize the chosen
objective, as the following objective summaries describe:
For nonlinear ARX models, the objective is either prediction-error minimization or simulation-error minimization, depending on whether the
Focusoption innlarxOptionsis"prediction"or"simulation".For Hammerstein-Wiener models, the objective is simulation-error-norm minimization.
See nlarxOptions and nlhwOptions for more information on how to configure the objective and search
method.
Version History
Introduced in R2023bSee Also
nlarx | idnlarx | nlarxOptions | nlhw | idnlhw | RegressionNeuralNetwork (Statistics and Machine Learning Toolbox) | fitrnet (Statistics and Machine Learning Toolbox) | dlnetwork (Deep Learning Toolbox) | network (Deep Learning Toolbox) | feedforwardnet (Deep Learning Toolbox) | trainingOptions (Deep Learning Toolbox) | evaluate
Topics
- Available Mapping Functions for Nonlinear ARX Models
- Cascade-Correlation Neural Networks
- Available Nonlinearity Estimators for Hammerstein-Wiener Models
- List of Deep Learning Layers (Deep Learning Toolbox)
- Define Custom Training Loops, Loss Functions, and Networks (Deep Learning Toolbox)
- Train and Apply Multilayer Shallow Neural Networks (Deep Learning Toolbox)
- Predict EV Battery Temperature Using Cascade-Correlation Model
