"experienc​es=allExpe​riences (saved.age​nt.Experie​nceBuffer)​"

Excuse me, when I am using the code "experiences=allExperiences (saved.agent.ExperienceBuffer)", I expect to get all experiences, but the system output is "experiences=empty 1x0 struct array without fields". May I ask why this situation has occurred.

Answer 1

[Anumeha]

Hi 铖,

I understand “allExperiences” is giving empty output, which would mean that the input it receives is empty somehow. Since the code and data isn’t provided, I was able to achieve a possible workflow using this documentation page.

https://www.mathworks.com/help/reinforcement-learning/ref/rl.replay.rlreplaymemory.allexperiences.html

The code I used is given below:

obsContinuous = rlNumericSpec([2 1],...
    LowerLimit=0,...
    UpperLimit=[1;5]);
obsDiscrete = rlFiniteSetSpec([1 2 3]);
obsInfo = [obsContinuous obsDiscrete];

actInfo = rlNumericSpec([2 1],...
    LowerLimit=0,...
    UpperLimit=[5;10]);

for i = 1:10
    experience(i).Observation = ...
        {obsInfo(1).UpperLimit.*rand(2,1) randi(3)};
    experience(i).Action = {actInfo.UpperLimit.*rand(2,1)};
    experience(i).NextObservation = ...
        {obsInfo(1).UpperLimit.*rand(2,1) randi(3)};
    experience(i).Reward = 10*rand(1);
    experience(i).IsDone = 0;
end

save('filename.mat', 'experience'); 

Now this is saved. So I cleared the experience variable, and reloaded it from the saved file.

load("filename.mat") 

buffer = rlReplayMemory(obsInfo,actInfo,5000); 
append(buffer,experience); 
experiences = allExperiences(buffer) 

This gives me the 10 experiences I saved.

Hope this helps.