How to load image sequence dataset which contain multiple subfolder then feed into RNN/LSTM model

Datasets

The dataset contain 3 class (Gesture_1, Gesture_2, Gesture_3). Each class has 10 samples which are stored in a sub folder of the class. All the samples are in jpg format. (frame1.jpg,frame2.jpg) .

Code

inputSize = [227 227 3];
numHiddenUnits = 128;
numClasses = 3;

%Load Dataset
imdsTrain = imageDatastore("D:\Database\GestureDataset","IncludeSubfolders",true,"LabelSource","foldernames");
[imdsTrain, imdsValidation] = splitEachLabel(imdsTrain,0.7);
augimdsTrain = augmentedImageDatastore(inputSize,imdsTrain);
augimdsValidation = augmentedImageDatastore(inputSize,imdsValidation);

layers = [ 
    sequenceInputLayer(inputSize,'Name','input')
    flattenLayer('Name','flatten')
    lstmLayer(numHiddenUnits,'OutputMode','last','Name','bi-lstm')
    fullyConnectedLayer(numClasses, 'Name','fc')
    softmaxLayer('Name','softmax')
    classificationLayer('Name','classification')];
    lgraph = layerGraph(layers);


opts = trainingOptions('sgdm', ...
    'MiniBatchSize',8, ...
    'MaxEpochs',16, ...
    'InitialLearnRate',0.0001, ...
    'Shuffle','every-epoch', ...
    'ValidationData',augimdsValidation, ...
    'ValidationFrequency',valFrequency, ...
    'Verbose',false, ...
    'Plots','training-progress');

    net = trainNetwork(augimdsTrain,lgraph,opts);

Problem

I'm able to load the dataset and train the model using the imageDatastore function, but the subfolder of each class will also consider as class using this method to load dataset.

How load the dataset which have multiple subfolder for each class, then feed into the model. (3 class, 10 samples each class)

Answer 1

[Aastha]

Hi @Jonathan,

I understand that you have a dataset organized in folders and subfolders according to the classes and the corresponding samples in the class which you want to load into MATLAB using an "imageDatastore".

To load a dataset having multiple subfolders per class you can create a "FileSet" object to specify the images to be read and explicitly pass the full paths to the images to the "imageDatastore".

For more information on specifying the list of files to be read, you may refer to the Examples section of the "imageDatastore" documentation linked below:

https://www.mathworks.com/help/matlab/ref/matlab.io.datastore.imagedatastore.html#butueui-2:~:text=datastore%20is%20shuffleable-,Examples,-collapse%20all

Once the "imageDatastore" object is created you can modify the "labels" argument using the dot notation and update the corresponding labels.

Kindly refer to the properties section of “imageDatastore” documentation for any queries on setting label parameters:

https://www.mathworks.com/help/matlab/ref/matlab.io.datastore.imagedatastore.html#butueui-3

Hope this helps!