Readmatrix in struct using parfor

Hi,

My for-loop that reads multiple .csv files (1 file per iteration) takes too much time. Therefore, I would like to increase the performance using parfor.

Using parfor and structs seems not to be straightforward. How can I adjust the following code to use a parfor?

Thanks.

% load the data from each file
% parfor i = 1:length(Files2Read)
for i = 1:length(Files2Read)
    
    % create fieldname for each file to read
    fld = ['File#', num2str(i) ];
    
    % load data
    Data.(fld) = readmatrix(Files2Read{i});
    
    % store additional information:
    Data.General.FieldName{i} = fld;
    
end

[Stephen23]

" How can I adjust the following code to use a parfor?"

Use actual indexing into a structure array rather than forcing a pseudo-index into the fieldname.

https://www.mathworks.com/help/matlab/matlab_prog/access-multiple-elements-of-a-nonscalar-struct-array.html

Answer 1

[Jan]

Fields names cannot contain the charcater '#' .

    fld = sprintf('File_%d', i);
    Data.(fld) = readmatrix(Files2Read{i});    

This should work.

AS Stephen told already, this would be nicer:

Data(i).Value = readmatrix(Files2Read{i});

or

Data.File(i) = readmatrix(Files2Read{i});   % Or File{i}

By the way, if the disk access is the bottleneck, parfor cannot help. Please post, if you code runs faster than with for.

[Konvictus177]

Thanks.

The code runs much faster using the changes in structure and parallel computing.

From 25 seconds down to 6 seconds. However, this is only true when the code is run the second time.

The first time I run the code it actually takes longer (32 seconds) because MATLAB is:

Starting parallel pool (parpool) using the 'local' profile ...

Connected to the parallel pool (number of workers: 6).

Lets say I want to build a standalone application and want matlab to start with parallel computing by default or do you see any other way to prevent the code from running longer each time the exe of the application is executed.

[Jan]

Starting the parallel pool takes some time, but remember, that reading the files might store their contents in a RAM cache also. Then optimizing the code for repeated reading has much less effects in real applications.

[Konvictus177]

Thanks.