Hi Danial,
In Reinforcement Learning Toolbox in MATLAB, different types of agents have different requirements and capabilities for handling observation spaces. The rlPPOAgent can handle more complex observation spaces, such as matrices, because it is designed for continuous action spaces and uses neural networks that can process multi-dimensional inputs. You can refer to this link for the documentation: rlPPOAgent
On the other hand, rlQAgent is typically designed for simpler, discrete action spaces and might have limitations regarding the observation space's structure. Specifically, rlQAgent expects the observation space to be a vector rather than a matrix.
To work around this limitation, you can reshape your matrix observation space into a vector. Here’s how you can do it:
- Convert the matrix to a vector before feeding it into the rlQAgent.
- Update the ObsInfo1 to reflect the new vector shape.
Why PPOAgent Can Handle Matrix Observations
The rlPPOAgent uses neural networks that can naturally handle multi-dimensional inputs, such as images or matrices. These networks can process the spatial structure of the data effectively. On the other hand, rlQAgent typically relies on simpler representations like tables or linear function approximators which are better suited for vector inputs.
Why QAgent Cannot Handle Matrix Observations
The rlQAgent is optimized for discrete action spaces and simpler observation spaces. Handling multi-dimensional observations would require more complex processing capabilities, which are not typically needed for the types of problems rlQAgent is designed to solve.
By reshaping your matrix observation space into a vector, you can work within the limitations of rlQAgent while still using your original data.
