Feed data into Neural Networks file-by-file

Currently for training my neural network, I need to load a huge dataset and it costs almost available RAM in the computer. That's why the training step (using GPU) is very slow. The pseudo-code is following:

Loading and pre-processing dataset (300 files)
Training Neural Network

So how can I don't have to load all at one time, and will be loaded in time the neural network is being trained, like this:

For file in Dataset
    Loading and pre-processing file
    Training Neural Network    

or

Training Neural network
At some epoch:
    Loading and Pre-processing a file in dataset
    Continue to train.
    

Does MATLAB currently support this feature?

Answer 1

[Joss Knight]

Datastores are designed for precisely this purpose. It may be that you're after an imageDatastore processed by a transform.