Simple Generative Adversarial Network

This MATLAB script demonstrates a simple Generative Adversarial Network (GAN) for multivariate function approximation.

This MATLAB script demonstrates a simple Generative Adversarial Network (GAN) which includes two feedforward artificial neural networks (ANNs) trained using the backpropagation algorithm. The GAN approximates the function:

f(x)=sum(x.^2,2), x=[x1,x2,x3,...,xk]

where k is the number of independent variables of the approximated function (equal to n_vars in the main script GAN_demo.m). The generator and discriminator ANNs of the GAN consist of a single hidden layer. This code is based on Chapter 6 of the following book:

Brownlee, J. (2019). Generative adversarial networks with python: deep learning generative models for image synthesis and image translation. Machine Learning Mastery.

which is also available as a tutorial at:

https://machinelearningmastery.com/how-to-develop-a-generative-adversarial-network-for-a-1-dimensional-function-from-scratch-in-keras/

The main script serves as a reference implementation to help users understand how GAN can be implemented and modify it for more advanced applications. As a generalization to the above tutorial, this script can handle more than one independent variables for the approximation function (up to k ).

No MATLAB toolbox is required to run this code, which is particularly useful for educational GAN prototypes or if you want fine-grained control over weight updates, learning rate, activation functions, etc. No dependencies on MATLAB’s Deep Learning Toolbox or any other toolboxes exist. It is transparent and easy to extend (ideal for GAN research and learning). The local functions used in this script are listed alphabetically at the end of the main script.

Note: Your results may vary given the stochastic nature of the algorithm or evaluation procedure, or differences in numerical precision. Consider running the example a few times and compare the average outcome.

This script is open-source. Feel free to use and modify it for research and educational purposes.

Features:

Uses ReLU activation function in the hidden layer of the ANNs involved (generator and discriminator).

Uses linear function for the output layer of the generator ANN and sigmoid function for the output layer of the discriminator ANN.

Gradient-based weight updates for training of the ANNs involved.

Implements He weight initialization of the ANNs involved. The biases are initialized as zeros.

Binary cross-entropy loss tracking across epochs for both ANNs involved.

Customizable network parameters, including:

Hidden layer size of generator ANN.

Hidden layer size of discriminator ANN.

Size of latent space (input space of the generator ANN)

Maximum number of training epochs.

Batch size (i.e. the data size per epoch used for training of the GAN)

Learning rate.