receiving different training results while running the same code

I ran the training of my RL model but forgot to save so i thought i would run the same script again

but i am getting a slightly changed response ?

shouldnt i get the same training results?

also what is relation b/w different sampling times of like actor and agent.

[Emmanouil Tzorakoleftherakis]

Can you clarify what you mean by different sample times between agent and actor? Each agent has its own sample time which indicates how often you want to do inference/get an action. Did you see different sample time option used for the actor?

[Sourabh]

okay so does it mean higher sample time of my agent better control or what ?

also in training if max steps is 100 does it mean the simulation is running for 100 sec / episode ?

[Emmanouil Tzorakoleftherakis]

max steps will depend on your agent sample time. If it's 100, it means thatthe total episode duration will be 100* ts where ts is the agent sample time.

Also, smaller sample time does not necessarily mean better control. As a rule of thumb, your sample time should only be as small as needed to get good results, not smaller than that to avoid wasting computational resources.

Answer 1

[Steven Lord]

Are random numbers involved in the process of creating or training your RL model? [My guess is most likely yes.] One way to check this would be to set the state of the random number generator to a known, fixed value using rng then run your code. Reset the generator to that same known, fixed value and run your code again.

rng(0, 'twister');
x = rand(1, 5)

x = 1×5

0.8147 0.9058 0.1270 0.9134 0.6324

y = rand(1, 5) % not the same as x

y = 1×5

0.0975 0.2785 0.5469 0.9575 0.9649

isequal(x, y)

ans = logical
   0

rng(0, 'twister');
y = rand(1, 5) % the same as x

y = 1×5

0.8147 0.9058 0.1270 0.9134 0.6324

isequal(x, y)

ans = logical
   1