How to use trainNetwork with transform datastore with multiple outputs?

my code create a audioDataStore, transform it to yamnet features (mel spectrum) and try to train it.

But i get the following error:

Input datastore returned more than one observation per row for network input 1.

The code:

net = yamnet;

DS = audioDatastore(FolderName, ...

'FileExtensions',{'.wav','.mp3'},"IncludeSubfolders",true,'LabelSource','foldernames');

DS.Labels = setcats(DS.Labels,cellstr(net.Layers(86).Classes));

TR = transform(DS,@(audio,info)preProcess(audio,info),"IncludeInfo",true);

options = trainingOptions("adam");

trainNetwork(TR,net.Layers,options)

function [data,info] = preProcess(audio,info)

data{1} = yamnetPreprocess(audio,info.SampleRate);

data{2} = repmat(info.Label,1,size(data{1},4));

end

Yamnet produce multiple observations for each audio file (for instance, for 10 seconds file it will create 96×64×1×19 matrix which are 19 observations. For some reason trainNetwork want only one observation, how do I fix it?

Answer 1

[jibrahim]

Hi Noam,

Check out this example:

Transfer Learning with Pretrained Audio Networks in Deep Network Designer

The example uses a transformed datastore as well. the transformation function is at the bottom:

function [data,info] = audioPreprocess(audioIn,info)
class = info.Label;
fs = info.SampleRate;
features = yamnetPreprocess(audioIn,fs);

numSpectrograms = size(features,4);

data = cell(numSpectrograms,2);
for index = 1:numSpectrograms
    data{index,1} = features(:,:,:,index);
    data{index,2} = class;
end
end

[Noam]

Thanks! It looks like to work.