Playing Pong® with deep reinforcement learning

Train a reinforcement learning agent to play a variation of Pong®

This example demonstrates a reinforcement learning agent playing a variation of the game of Pong® using Reinforcement Learning Toolbox™. You will follow a command line workflow to create a DDPG agent in MATLAB®, set up hyperparameters and then train and simulate the agent.

The following two scripts can be used to train or simulate the agent:
train_agent.m - script for creating and training a reinforcement learning agent
play_agent.m - script for playing the game

The following scripts are used to create the environment:
Environment.m - class for modeling the game
Visualizer.m - class for animation functions

Environment
The Environment for the game is a two dimensional space with a ball and a paddle. The ball starts with an initial velocity and moves around in the environment. The walls restrict the ball from moving outside the environment and also transfers some momentum to the ball on collision. For this reason there is a slight velocity change whenever the ball collides an object. The paddle is located at the bottom half and can move left to right to prevent the ball falling below.

Agent
A Deep Deterministic Policy Gradient (DDPG) reinforcement learning agent is used in this example. The agent learns to hit the ball by observing the following states in the environment:

1. x, y positions of the ball
2. x, y velocities of the ball
3. x position of the paddle
4. x velocity of the paddle
5. Action values from the last time step

The action of the agent is the force applied on the paddle in the x direction.

Train
To create an agent and run the training, open and run the train_agent.m script.

Play
To view a pre-trained agent playing the game, use the script play_agent.m.