predictAndUpdateState
Class: dlhdl.Workflow
Namespace: dlhdl
Predict responses by using a trained and deployed recurrent neural network and update the deployed network state
Since R2022b
Syntax
Description
predicts responses for data in Y
= predictAndUpdateState(workflowObject
,sequences
)sequences
using the deployed network and
updates the network state. The method does not initialize the network state before
running. This method supports recurrent neural networks only. The specified network must
have at least one recurrent layer, such as an LSTM layer or a custom layer with state
parameters.
[
predicts responses and updates the network state with one or more arguments specified by
optional name-value pair arguments.Y
,performance
] = predictAndUpdateState(workflowObject
,sequences
,Name,Value
)
Input Arguments
workflowObject
— Deep learning network deployment options
dlhdl.Workflow
object
Deep learning network deployment options, specified as a
dlhdl.Workflow
object.
sequences
— Sequence or time series data
numeric array
For numeric array input, the dimensions of the numeric arrays containing the sequences depend on the type of data.
Input | Description |
---|---|
Vector sequences | c-by-s matrices, where c is the number of features of the sequences and s is the sequence length. |
1-D image sequences | h-by-c-by-s arrays, where h and c correspond to the height and number of channels of the images, respectively, and s is the sequence length. |
2-D image sequences | h-by-w-by-c-by-s arrays, where h, w, and c correspond to the height, width, and number of channels of the images, respectively, and s is the sequence length. |
3-D image sequences | h-by-w-by-d-by-c-by-s, where h, w, d, and c correspond to the height, width, depth, and number of channels of the 3-D images, respectively, and s is the sequence length. |
The dimensions of the sequence data must correspond to the table.
Data Types: single
| double
| int8
| int16
| int32
| int64
| uint8
| uint16
| uint32
| uint64
Name-Value Arguments
Specify optional pairs of arguments as
Name1=Value1,...,NameN=ValueN
, where Name
is
the argument name and Value
is the corresponding value.
Name-value arguments must appear after other arguments, but the order of the
pairs does not matter.
Before R2021a, use commas to separate each name and value, and enclose
Name
in quotes.
Example:
Profile
— Flag that returns profiling results
"off" (default) | "on"
Flag to return profiling results, for the deep learning network deployed to the
target board, specified as "off"
or "on"
.
Example: "Profile", "On"
Output Arguments
Y
— Predicted responses
numeric array
Predicted responses, returned as a numeric array. The format of
Y
depends on the type of task.
This table describes the format for regression problems.
Task | Format |
---|---|
2-D image regression |
|
3-D image regression |
|
Sequence-to-one regression | N-by-R matrix, where N is the number of sequences and R is the number of responses |
Sequence-to-sequence regression | N-by-R matrix, where N is the number of sequences and R is the number of responses |
Feature regression | N-by-R matrix, where N is the number of observations and R is the number of responses |
For sequence-to-sequence regression problems with one observation,
sequences
can be a matrix. In this case, Y
is a matrix of responses.
If the output layer of the network is a classification layer, then
Y
is the predicted classification scores. This table describes
the format of the scores for classification tasks.
Task | Format |
---|---|
Image classification | N-by-K matrix, where N is the number of observations and K is the number of classes |
Sequence-to-label classification | |
Feature classification |
performance
— Deployed network performance data
table
Deployed network performance data, returned as an
N
-by-5 table, where
N
is the number of layers in the network.
This method returns performance only when the Profile
name-value
argument is set to 'on'
. To learn about the data in the performance
table, see Profile Inference Run.
Examples
Run Sequence Forecasting on FPGA by Using Deep Learning HDL Toolbox
This example shows how to create, compile, and deploy a long short-term memory (LSTM) network trained on waveform data by using the Deep Learning HDL Toolbox™ Support Package for Xilinx FPGA and SoC. Use the deployed network to predict future values by using open-loop and closed-loop forecasting. Use MATLAB® to retrieve the prediction results from the target device.
Waveform Data Network
The network attached to this example was trained using the Time Series Forecasting Using Deep Learning. This example uses the WaveformData.mat
data set, which contains 2000 synthetically generated waveforms of varying lengths with three channels. This example uses a trained LSTM network to forecast future values of the waveforms given the values from the previous time steps using both closed loop and open loop forecasting.
Prerequisites
Xilinx® Zynq® Ultrascale+™ ZCU102 SoC development kit
Deep Learning HDL Toolbox™ Support Package for Xilinx FPGA and SoC
Deep Learning Toolbox™
Deep Learning HDL Toolbox™
Load the Pretrained Network
To load the LSTM network enter:
load WaveformForcastingNet
Use the analyzeNetwork
function to obtain information about the network layers. the function returns a graphical representation of the network that contains detailed parameter information for every layer in the network.
analyzeNetwork(net)
Define FPGA Board Interface
Define the target FPGA board programming interface by using the dlhdl.Target
object. Specify that the interface is for a Xilinx board with an Ethernet interface.
To create the target object, enter:
hTarget = dlhdl.Target('Xilinx','Interface','Ethernet');
To use the JTAG interface, install Xilinx™ Vivado™ Design Suite 2022.1. To set the Xilinx Vivado toolpath, enter:
hdlsetuptoolpath('ToolName', 'Xilinx Vivado', 'ToolPath', 'C:\Xilinx\Vivado\2022.1\bin\vivado.bat'); hTarget = dlhdl.Target('Xilinx','Interface','JTAG');
Prepare Network for Deployment
Prepare the network for deployment by creating a dlhdl.Workflow
object. Specify the network and the bitstream name. Ensure that the bitstream name matches the data type and the FPGA board. In this example the target FPGA board is the Xilinx ZCU102 SOC board. The bitstream uses a single data type.
hW = dlhdl.Workflow('network', net, 'Bitstream', 'zcu102_lstm_single','Target',hTarget);
Tu run the example on the Xilinx ZC706 board, enter:
hW = dlhdl.Workflow('Network', snet, 'Bitstream', 'zc706_lstm_single','Target',hTarget);
Compile the LSTM Network
Run the compile
method of the dlhdl.Workflow
object to compile the network and generate the instructions, weights, and biases for deployment. The total number of frames exceeds the default value of 30. Set the InputFrameNumberLimit
name-value argument to 1000
to run predictions in chunks of 1000 frames to prevent timeouts.
dn = compile(hW,'InputFrameNumberLimit',1000)
### Compiling network for Deep Learning FPGA prototyping ... ### Targeting FPGA bitstream zcu102_lstm_single. ### The network includes the following layers: 1 'sequenceinput' Sequence Input Sequence input with 3 dimensions (SW Layer) 2 'lstm' LSTM LSTM with 128 hidden units (HW Layer) 3 'fc' Fully Connected 3 fully connected layer (HW Layer) 4 'regressionoutput' Regression Output mean-squared-error with response 'Response' (SW Layer) ### Notice: The layer 'sequenceinput' with type 'nnet.cnn.layer.ImageInputLayer' is implemented in software. ### Notice: The layer 'regressionoutput' with type 'nnet.cnn.layer.RegressionOutputLayer' is implemented in software. ### Compiling layer group: lstm.wi ... ### Compiling layer group: lstm.wi ... complete. ### Compiling layer group: lstm.wo ... ### Compiling layer group: lstm.wo ... complete. ### Compiling layer group: lstm.wg ... ### Compiling layer group: lstm.wg ... complete. ### Compiling layer group: lstm.wf ... ### Compiling layer group: lstm.wf ... complete. ### Compiling layer group: fc ... ### Compiling layer group: fc ... complete. ### Allocating external memory buffers: offset_name offset_address allocated_space _______________________ ______________ _________________ "InputDataOffset" "0x00000000" "16.0 kB" "OutputResultOffset" "0x00004000" "16.0 kB" "SchedulerDataOffset" "0x00008000" "612.0 kB" "SystemBufferOffset" "0x000a1000" "20.0 kB" "InstructionDataOffset" "0x000a6000" "4.0 kB" "FCWeightDataOffset" "0x000a7000" "272.0 kB" "EndOffset" "0x000eb000" "Total: 940.0 kB" ### Network compilation complete.
dn = struct with fields:
weights: [1×1 struct]
instructions: [1×1 struct]
registers: [1×1 struct]
syncInstructions: [1×1 struct]
constantData: {}
ddrInfo: [1×1 struct]
resourceTable: [6×2 table]
Program Bitstream onto FPGA and Download Network Weights
To deploy the network on the Xilinx ZCU102 SoC hardware, run the deploy
function of the dlhdl.Workflow
object. This function uses the output of the compile
function to program the FPGA board by using the programming file. It also downloads the network weights and biases. The deploy
function starts programming the FPGA device and displays progress messages, and the required time to deploy the network.
deploy(hW)
### Programming FPGA Bitstream using Ethernet... ### Attempting to connect to the hardware board at 192.168.1.101... ### Connection successful ### Programming FPGA device on Xilinx SoC hardware board at 192.168.1.101... ### Copying FPGA programming files to SD card... ### Setting FPGA bitstream and devicetree for boot... # Copying Bitstream zcu102_lstm_single.bit to /mnt/hdlcoder_rd # Set Bitstream to hdlcoder_rd/zcu102_lstm_single.bit # Copying Devicetree devicetree_dlhdl.dtb to /mnt/hdlcoder_rd # Set Devicetree to hdlcoder_rd/devicetree_dlhdl.dtb # Set up boot for Reference Design: 'AXI-Stream DDR Memory Access : 3-AXIM' ### Rebooting Xilinx SoC at 192.168.1.101... ### Reboot may take several seconds... ### Attempting to connect to the hardware board at 192.168.1.101... ### Connection successful ### Programming the FPGA bitstream has been completed successfully. ### Resetting network state. ### Loading weights to FC Processor. ### 50% finished, current time is 10-Dec-2023 14:33:24. ### FC Weights loaded. Current time is 10-Dec-2023 14:33:24
Test Network
Prepare the test data for prediction. Normalize the test data using the statistics calculated from the training data. To forecast the values of future time steps of a sequence, specify the targets as the test sequences with values shifted by one time step. In other words, at each time step of the input sequence, the LSTM network learns to predict the value of the next time step. The predictors as the test sequences without the final time step.
load WaveformData.mat data = cellfun(@(x)x',data,UniformOutput=false); numChannels = size(data{1},1); numObservations = numel(data); idxTrain = 1:floor(0.9*numObservations); idxTest = floor(0.9*numObservations)+1:numObservations; dataTrain = data(idxTrain); dataTest = data(idxTest); for n = 1:numel(dataTrain) X = dataTrain{n}; XTrain{n} = X(:,1:end-1); TTrain{n} = X(:,2:end); end muX = mean(cat(2,XTrain{:}),2); sigmaX = std(cat(2,XTrain{:}),0,2); muT = mean(cat(2,TTrain{:}),2); sigmaT = std(cat(2,TTrain{:}),0,2); for n = 1:size(dataTest,1) X = dataTest{n}; XTest{n} = (X(:,1:end-1) - muX) ./ sigmaX; TTest{n} = (X(:,2:end) - muT) ./ sigmaT; end
Make predictions using the test data.
YTest = hW.predict(XTest{1},Profile ='on');
### Resetting network state. ### Finished writing input activations. ### Running a sequence of length 115. Deep Learning Processor Profiler Performance Results LastFrameLatency(cycles) LastFrameLatency(seconds) FramesNum Total Latency Frames/s ------------- ------------- --------- --------- --------- Network 33810 0.00015 115 3916501 6459.8 memSeparator_0 95 0.00000 memSeparator_3 184 0.00000 lstm.wi 7580 0.00003 lstm.wo 7569 0.00003 lstm.wg 7549 0.00003 lstm.wf 7649 0.00003 lstm.sigmoid_1 222 0.00000 lstm.sigmoid_3 224 0.00000 lstm.tanh_1 244 0.00000 lstm.sigmoid_2 224 0.00000 lstm.multiplication_2 294 0.00000 lstm.multiplication_1 364 0.00000 lstm.c_add 288 0.00000 lstm.tanh_2 228 0.00000 memSeparator_2 174 0.00000 lstm.multiplication_3 294 0.00000 fc 460 0.00000 memSeparator_1 168 0.00000 * The clock frequency of the DL processor is: 220MHz
To evaluate the accuracy, calculate the root mean squared error (RMSE) between the predictions and the target for each test sequence.
for i = 1:size(YTest,1) rmse(i) = sqrt(mean((YTest(i) - TTest{1}(i)).^2,"all")); end
Visualize the errors in a histogram. Lower values indicate greater accuracy.
figure histogram(rmse) xlabel("RMSE") ylabel("Frequency")
Calculate the mean RMSE over all test observations.
mean(rmse)
ans = single
0.8385
Forecast Future Time Steps
To forecast the values of multiple future time steps, when given an input time series or sequence, use the predict
function. This function predicts time steps one at a time and updates the network state at each prediction. For each prediction, use the previous prediction as the input to the function.
Visualize one of the test sequences in a plot.
idx = 2; X = XTest{idx}; T = TTest{idx}; figure stackedplot(X',DisplayLabels="Channel " + (1:numChannels)) xlabel("Time Step") title("Test Observation " + idx)
Open-Loop Forecasting
Open-loop forecasting predicts the next time step in a sequence using only the input data. When making predictions for subsequent time steps, you collect the true values form your data source and use those as input. For example, suppose that you want to predict the value for time step of a sequence by using data collected in time steps 1 through . To make predictions for time step , wait until you record the true value for time step and use that value as input to make the next prediction. Use open-loop forecasting when you have true values to provide to the network before making the next prediction.
Initialize the network state by resetting the state using the resetState
function, then make an initial prediction using the first few time steps of the input data. Update the network state by using the first 75 time steps of the input data.
resetState(hW)
offset = 75;
[~,~] = hW.predict(X(:,1:offset),KeepState=true,Profile='on');
### Resetting network state. ### Finished writing input activations. ### Running a sequence of length 75. Deep Learning Processor Profiler Performance Results LastFrameLatency(cycles) LastFrameLatency(seconds) FramesNum Total Latency Frames/s ------------- ------------- --------- --------- --------- Network 33730 0.00015 75 2554104 6460.2 memSeparator_0 95 0.00000 memSeparator_3 184 0.00000 lstm.wi 7620 0.00003 lstm.wo 7449 0.00003 lstm.wg 7549 0.00003 lstm.wf 7659 0.00003 lstm.sigmoid_1 222 0.00000 lstm.sigmoid_3 224 0.00000 lstm.tanh_1 254 0.00000 lstm.sigmoid_2 224 0.00000 lstm.multiplication_2 344 0.00000 lstm.multiplication_1 294 0.00000 lstm.c_add 288 0.00000 lstm.tanh_2 228 0.00000 memSeparator_2 174 0.00000 lstm.multiplication_3 294 0.00000 fc 420 0.00000 memSeparator_1 208 0.00000 * The clock frequency of the DL processor is: 220MHz
To forecast further predictions, loop over time steps and update the network state by using the predict
function and setting the KeepState
name-value argument to true
. Forecast values for the remaining time steps of the test observation by looping over the time steps of the input data and using them as input to the network. The first prediction is the value that corresponds to the time step offset + 1
.
numTimeSteps = size(X,2);
numPredictionTimeSteps = numTimeSteps - offset;
Y = hW.predict(X(:,offset+1:offset+numPredictionTimeSteps),KeepState=true,Profile='on');
### Finished writing input activations. ### Running a sequence of length 116. Deep Learning Processor Profiler Performance Results LastFrameLatency(cycles) LastFrameLatency(seconds) FramesNum Total Latency Frames/s ------------- ------------- --------- --------- --------- Network 33770 0.00015 116 3947829 6464.3 memSeparator_0 95 0.00000 memSeparator_3 174 0.00000 lstm.wi 7570 0.00003 lstm.wo 7549 0.00003 lstm.wg 7589 0.00003 lstm.wf 7609 0.00003 lstm.sigmoid_1 222 0.00000 lstm.sigmoid_3 224 0.00000 lstm.tanh_1 274 0.00000 lstm.sigmoid_2 224 0.00000 lstm.multiplication_2 334 0.00000 lstm.multiplication_1 294 0.00000 lstm.c_add 308 0.00000 lstm.tanh_2 238 0.00000 memSeparator_2 184 0.00000 lstm.multiplication_3 294 0.00000 fc 420 0.00000 memSeparator_1 168 0.00000 * The clock frequency of the DL processor is: 220MHz
Compare the predictions with the target values.
figure t = tiledlayout(numChannels,1); title(t,"Open Loop Forecasting with LSTM layer") for i = 1:numChannels nexttile plot(T(i,:)) hold on plot(offset:numTimeSteps,[T(i,offset) Y(i,:)],'--') ylabel("Channel " + i) end xlabel("Time Step") nexttile(1) legend(["Input" "Forecasted"])
Closed-Loop Forecasting
Closed-loop forecasting predicts subsequent time steps in a sequence by using the previous predictions as input. In this case, the model does not require the true values to make the prediction. For example, suppose that you want to predict the value for time steps through of the sequence by using data collected in time steps 1 through . To make predictions for time step , use the predicted value for time step as input. Use closed-loop forecasting to forecast multiple subsequent time steps or when you do not have true values to provide to the network before making the next prediction.
Initialize the network state by resetting the state using the resetState
function, then make an initial prediction, Z,
using the first few time steps of the input data. Update the network state by using the first 75 time steps of the input data.
resetState(hW)
[Z, ~] = predict(hW,X,KeepState=true,Profile='on');
### Resetting network state. ### Finished writing input activations. ### Running a sequence of length 191. Deep Learning Processor Profiler Performance Results LastFrameLatency(cycles) LastFrameLatency(seconds) FramesNum Total Latency Frames/s ------------- ------------- --------- --------- --------- Network 33717 0.00015 191 6502666 6462.0 memSeparator_0 95 0.00000 memSeparator_3 174 0.00000 lstm.wi 7580 0.00003 lstm.wo 7548 0.00003 lstm.wg 7458 0.00003 lstm.wf 7649 0.00003 lstm.sigmoid_1 221 0.00000 lstm.sigmoid_3 224 0.00000 lstm.tanh_1 244 0.00000 lstm.sigmoid_2 224 0.00000 lstm.multiplication_2 294 0.00000 lstm.multiplication_1 364 0.00000 lstm.c_add 288 0.00000 lstm.tanh_2 238 0.00000 memSeparator_2 184 0.00000 lstm.multiplication_3 294 0.00000 fc 470 0.00000 memSeparator_1 168 0.00000 * The clock frequency of the DL processor is: 220MHz
To forecast further predictions, loop over time steps and update the network state by using the predict
function and setting the KeepState
name-value argument to true
. Forecast the next 200 time steps by iteratively passing the previously predicted value to the network. Because the network does not require the input data to make any further predictions, you can specify any number of time steps to forecast.
numPredictionTimeSteps = 200;
Xt = Z(:,end);
Y = zeros(numChannels,numPredictionTimeSteps);
fprintf("Run %d predictions:\n",numPredictionTimeSteps);
Run 200 predictions:
for t = 1:numPredictionTimeSteps fprintf("."); [Y(:,t),~] = predict(hW,Xt,KeepState=true); Xt = Y(:,t); end
Visualize the forecasted values in a plot.
offset = size(X,2); numTimeSteps = offset + numPredictionTimeSteps; figure t = tiledlayout(numChannels,1); title(t,"Closed Loop Forecasting with LSTM layer") for i = 1:numChannels nexttile plot(T(i,1:offset)) hold on plot(offset:numTimeSteps,[T(i,offset) Y(i,:)],'--') ylabel("Channel " + i) end xlabel("Time Step") nexttile(1) legend(["Input" "Forecasted"])
Closed-loop forecasting allows you to forecast an arbitrary number of time steps, but can be less accurate when compared to open-loop forecasting because the network does not have access to the true values during the forecasting process.
Version History
Introduced in R2022b
See Also
activations
| compile
| deploy
| predict
| resetState
MATLAB 명령
다음 MATLAB 명령에 해당하는 링크를 클릭했습니다.
명령을 실행하려면 MATLAB 명령 창에 입력하십시오. 웹 브라우저는 MATLAB 명령을 지원하지 않습니다.
Select a Web Site
Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .
You can also select a web site from the following list:
How to Get Best Site Performance
Select the China site (in Chinese or English) for best site performance. Other MathWorks country sites are not optimized for visits from your location.
Americas
- América Latina (Español)
- Canada (English)
- United States (English)
Europe
- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)
- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- United Kingdom (English)