필터 지우기
필터 지우기

Using a neural network with a dropout layer in Simulink

조회 수: 7 (최근 30일)
Emirhan INANC
Emirhan INANC 2022년 10월 23일
댓글: Emirhan INANC 2022년 10월 27일
I am trying to add a dropout layer to my LSTM structure to prevent overfitting. Then, I am using this LSTM model in Simulink Environment, using Stateful Predict block. Unfortunately, this block is very slow on forward passing my network (it takes too much time for me, even with the accelerator). Therefore, I had to use my own forward pass code, but to do that I need to know which neurons are disabled in my dropout layer. Is there a way for me to obtain this information?
Note: There is no learning happening in dropout layer, but the simulation results are affecting my training process as well, so i need to disable corresponding neurons.

채택된 답변

Arkadiy Turevskiy
Arkadiy Turevskiy 2022년 10월 24일
Hi,
I'd like to mention a couple of things.
  1. Please take a look at the doc. It says: "At training time, the layer randomly sets input elements to zero ..and helps prevent the network from overfitting. .. At prediction time, the output of the layer is equal to its input.". When you place trained LSTM in Simulink, there is no training happening, only inference. Therefore, the output of the droput layer should just be equal to its input.
  2. Regarding slow speed of Stateful Predict bloc. By default it will run in interpreted mode, meaning it call into MATLAB at each execution. You can improve performance by simulating using code generation. This requires you to do a few steps that are described here. Please see if that helps you improve the speed.
Hope this helps.
Arkadiy
  댓글 수: 1
Emirhan INANC
Emirhan INANC 2022년 10월 27일
Thanks for your answer, but I have some question marks in my mind about my problem.
  1. Actually, the problem is, in the training time, also forward pass is needed to calculate the loss of the network. The only way for me to calculate this forward pass is to run the simulation once. Because my dataset is created during the training process. I run the simulation and obtain my input, output, and truth for my network. Then I use the collected data in my training process.
  2. Thanks for your advice. I tried this method after your suggestion. Unfortunately, changing the language of code generation from C to C++ didn't make the simulation faster at all. I want to mention that my network is not deep (1 hidden layer, 128 neurons). I think this method is just for the " predict " block, not for the " stateful predict " block, where we use recurrent neural networks.

댓글을 달려면 로그인하십시오.

추가 답변 (0개)

카테고리

Help CenterFile Exchange에서 Deep Learning Toolbox에 대해 자세히 알아보기

제품


릴리스

R2022a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by