It would maybe be a solution to add output layers to the network and display their result, but it's tedious.
Debug mode for RL agent networks
조회 수: 10 (최근 30일)
이전 댓글 표시
Is there any way to enter debug mode, to see what is happening inside the RL agent nets while training is running?
Some of my layers output NaN and I would like to know which one. I would also like to monitor to outputs of each layer.
Thanks in advance!
Nicolas
채택된 답변
Shantanu Dixit
2024년 7월 19일
편집: Shantanu Dixit
2024년 7월 19일
Hi Nicolas,
It is my understanding that you want to monitor the outputs of each layer and debug the RL agent nets while the training is running.
Similar to analyzing deep learning networks you can call the forward method for each layer of the agent's network to analyze the corresponding output during the training.
[Y1,...,YK] = forward(___,'Outputs',layerNames)
here 'layerNames' correspond to a string array, with 'layerNames(k)' representing kth layer of the agent's network, the corresponding outputs are stored in 'Yk'
Briefly you can follow the below steps:
- Extract layer names from the actor network into a string array 'layerNames'
- Convert Observation Buffer to dlarray for processing into the network
- Forward Pass Through Each Layer using the forward method.
Refer the below code for monitoring the layer outputs corresponding to one observation after updating the actor
%% actorNetwork refers to the DNN of the actor
%% for one observation
dlX = dlarray(observationBuffer(:,:,1), 'CB'); %% format in which the network takes the input
[Y1, Y2, Y3, Y4, Y5, Y6] = forward(actorNetwork, dlX, 'Outputs', layerNames);
layerOutputs = {Y1, Y2, Y3, Y4, Y5, Y6};
for i = 1:numel(layerOutputs)
% disp(['Layer ', layerNames(i), ' output:']);
if any(isnan(extractdata(layerOutputs{i})), 'all')
disp(['Layer ', num2str(i), ' output contains NaNs']); %% check for NaNs
end
end
For a better understanding on forward pass and the custom training loop procedure, refer to the following MathWorks documentation
추가 답변 (0개)
참고 항목
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!