combine files with the same names that are located in different directories

Dear Matlab Experts,

I have several subdirectories in my parent folder ( named in the numeric order 1, 2, 3 etc) each of which includes several csv files. I need a handy code that combines all csv files of the same name located in different directories. For example, I have 4 files with the name 190.csv in directories 1 , 2, 3 and 6. (please note that directories 4 and 5 do not have a file with the name of 190.csv) I need to combine these files. Also my csv files are basically matrices. so my preference is to combine the as follows :

[190.csv (directory 1) ; 190.csv (directory 2); 190.csv (directory 3); 190.csv (directory 6)]

Can you please help me figuring out what to do? Please keep this in mind that in general, I do not know which directories have the target file. So I need to find the target files first and then combine them. So my question turns down to the possibility of merging several csv files that are storing different matrices into one csv file

Your helps is really appreciated. Thanks

Answer 1

[KSSV]

You have to search for the file you want (say 190.csv) every where in the computer, once you know the locations of the file present, you can append them. You may use dos search command from MATLAB to know the locations of the given file.

Have a look in the following link:

http://in.mathworks.com/matlabcentral/answers/32038-can-you-use-dir-to-list-files-in-subfolders

[Homayoon]

Thank you so much, but I have not quite got it. Can you please more specific?

[Homayoon]

Also it is not clear to me if what you suggested merge the files in the order of interest (As mentioned in the body of the question).

Answer 2

[Walter Roberson]

Grab https://www.mathworks.com/matlabcentral/fileexchange/32226-recursive-directory-listing-enhanced-rdir and https://www.mathworks.com/matlabcentral/fileexchange/47434-natural-order-filename-sort

projectdir = 'TheParentFolder';
outdir = 'DirToPutOutputFiles';

dinfo = rdir( fullfile(projectdir, '*/*.csv') );

allnames = natsortfiles( {dinfo.name} );
num_all_files = length(allnames);

basenames = cell(num_all_files, 1);
for K = 1 : num_all_files
  [~, basenames{K}, ~] = fileparts(allnames{K});
end

[unique_basenames, ~, name_idx] = unique(basenames);
num_unique_basenames = length(unique_basenames);

for K = 1 : num_unique_basenames
  this_basename = unique_basenames{K};
  outfile = fullfile( outdir, [this_basename '.csv'] );
  matches_this_base = name_idx == K;
  infiles = allnames(matches_this_base);

    %At this point, process all of the (directory-qualified) filenames in infiles,
    %which will be in order, sending the output to the file designated by outfile

end

I did not fill in the code for the actual merging of the files because the details would depend upon whether there is a common header that needs to be output only once, and the details would also depend upon whether you need the outputs to be exactly the same as the input, or only numerically the same. For example if a particular field happened to have '- 173434324324.12' complete with a space after the minus sign, then do you need exactly that output, or would it be okay if the minus sign got moved to be beside the number, or would you want -1.7343432432412e+11 output, or would you want -173434324324.1199951171875 output? (That is the exact number that would be stored for -173434324324.1 )

If there are not headers and you want the files to be copied exactly and you know for sure that the last useful input line of each ends in a newline, then you can take advantage of operating system operations to append the files instead of having to read and write the data inside MATLAB. For example (MS Windows only)

   cmd = ['copyfile "', strjoin(infiles, '"+"'), '" ', outfile);
   system(cmd)