Using datastore for a matrix

Hi

I am trying to set up a datastore in order to train a feedforward neural network with a trainingset that is too large for my workspace.

I have a training set, X, that is 80002x10000 which I stored in a .mat file called ¨TrainingData'. I am now trying to set up a datastore so I can load mini batches and train on those, but I can't get it working.

I have tried datastore and arrayDatastore.

datastore cannot determine the datastore type for the specified location and arrayDatastore just output the name of the .mat-file whenever i read, i.e.

>> read(ds)

ans =

1×1 cell array

{'X.mat'}

What am I doing wrong here?

EDIT

I just read that the arrayDatastore is only for in-memory data which makes it useless in my case

Thanks

Answer 1

[Catalytic]

If you're not going to store your data in memory, then the training samples have to be split up into separate .mat files. You could then use a fileDatastore to read from and manage them.

[Chuck Noise]

Thanks @Catalytic. I'll try that :)

[Catalytic]

A TallDatastore might also be appropriate.