put a function inside loop and save as matrix and csv

I have the following main script that pass value to a function. I would like to import different csv using function within a loop and output as summarize all data as matrix to be printed in an xls file . Here are my progress code so far, please help in matrix creationa and xls print. Thanks

%%main script-reads diff csv
clc
clear all
km=csvread('Bus_Data1.csv');
A=0
[Result1,maxa]=csvimport(km)

km=csvread('Bus_Data2.csv');
A=4
[Result1,maxa]=csvimport(km)

km=csvread('Bus_Data3.csv');
A=5
[Result1,maxa]=csvimport(km)

km=csvread('Bus_Data4.csv');
A=9
[Result1,maxa]=csvimport(km)

Here is the function code

    %%function to calculate output of main m
    function [Result1,maxa]=csvimport(fileName)

    A=fileName(1:5,1);
    maxa=max(A);
    Result1=A;
    end

[Rena Berman]

(Answers Dev) Restored edit

Answer 1

[Stephen23]

Simpler and much more efficient than ugly dynamic variable names:

S = dir('Bus_Data*.csv');
N = numel(S);
R = cell(1,N);
M = cell(1,N);
for k = 1:N
    km = csvread(S(k).name);
    [R{k},M{k}] = csvimport(km);
end

If you want the files loaded in numeric order then download my FEX submission natsortfiles.

[joms]

i tried this but it gives N=0. it doesnt read the csv files. how can i point to specific folder where i want csv to be read.

Index exceeds matrix
dimensions.

Error in csvimport (line 3)
A=fileName(1:5,1);

Error in main (line 10)
    [R{k},M{k}] =
    csvimport(S(k).name)

[Stephen23]

"how can i point to specific folder where i want csv to be read."

Use fullfile and the directory path:

D = 'directory where the files are';
S = dir(fullfile(D,'Bus_Data*.csv'));
N = numel(S);
R = cell(1,N);
M = cell(1,N);
for k = 1:N
    km = csvread(fullfile(D,S(k).name));
    [R{k},M{k}] = csvimport(km);
end

[Shaula Garibbo]

Hi Stephen,

I'm having a similar problem but I'm receiving the error message:

"Error using csvimport (line 68)

The first argument to csvimport must be a valid .csv file

Error in concat_col (line 14)

[R{k}, M{k}] = csvimport(C);"

Here is my code:

P = 'C:filepath';
S = dir(fullfile(P,'*.csv'));
N = numel(S);

R = cell(1,N);
M = cell(1,N);

for k = 1:N
   C = csvread(fullfile(P,S(k).name));
   [R{k}, M{k}] = csvimport(C);
end

The .csv files all consist of two numerical columns of varying lengths.

Do you have any advice?

Thanks

[Stephen23]

@Shaula Garibbo : I guess that you are using FEX 23573 csvimport, in which case that error message occurs whenever the input is not a character array:

if ~ischar( fileName )

So you should check if the input really is a character vector (e.g. and NOT a string, if you are using MATLAB version >R2016b). Note that 'C:filepath' is not a valid path.

I strongly recommend that you use one of the inbuilt file importing functions:

https://www.mathworks.com/help/matlab/import_export/supported-file-formats.html

[Shaula Garibbo]

Thank you. I'll try a different approach as per your suggestion. (I know 'filepath' isn't valid, it's just that the filepath is ridiculously long so I thought I'd delete it to make the comment shorter - sorry, should have said).

Answer 2

[Krithika A]

d = dir('Bus_Data*.csv');   % dir struct of all pertinent .csv files
n = length(d);        % how many there were
for i = 1:n
  data = csvread(d(i).name);
 [Result1,maxa] = csvimport(data)
 eval(['Result',num2str(i),'=Result1;'])
 eval(['maxa',num2str(i),'=maxa;'])
end

Eval isn't exactly efficient, but it's OK with you want the files separate.

[Stephen23]

"Eval isn't exactly efficient"

It is very inefficient, as the MATLAB documentation warns.

"but it's OK with you want the files separate."

It is not "OK": it is slow, buggy, obfuscated, and complex. In fact the MATLAB documentation specifically advises against doing this: "A frequent use of the eval function is to create sets of variables such as A1, A2, ..., An, but this approach does not use the array processing power of MATLAB and is not recommended. The preferred method is to store related data in a single array." Beginners who use eval force themselves into writing slow, complex, obfuscated, buggy code that is hard to debug.

All MATLAB experts will advise against doing this. Read more about why here:

https://www.mathworks.com/matlabcentral/answers/304528-tutorial-why-variables-should-not-be-named-dynamically-eval

[joms]

thanks for the answer. tried it but theres no output.

[Krithika A]

Haha a bit touchy about the eval function?

But thank you, I didn't know that about eval. Will try to avoid for future.

[joms]

itried your code and it works but i need result of all loops saved in a matrix . how can you do that. also the current directory in default /Users/XXX/Documents/MATLAB how can you change that to the folder where an main m file is located. Thanks

[Krithika A]

So the way I save into a matrix is using this:

files = dir('*.txt'); % You would use .csv
x_raw = [];
for q = 1:length(files);
    data = importfileB(files(q).name); % I made importfileB function myself
    x_raw = [x_raw, data]; % Puts data into a matrix
end

You'll need to adjust this yourself to fit your data structure. If it works, that is.

[joms]

i added some of the codes in your code it works pretty well now. my only problem is change to the directory of the m file. it current reads csv files on /Users/XXX/Documents/MATLAB . thanks for your help

%%main
clear all
clc

d = dir('Bus_Data*.csv');   % dir struct of all pertinent .csv files
n = length(d);       % how many there were
for i = 1:n
  data = csvread(d(i).name);
 [Result1,maxa] = csvimport(data);
 eval(['Result1',num2str(i),'=Result1;'])
 eval(['maxa',num2str(i),'=maxa;'])

   mat(i,:)=Result1;

end
mat=mat

[Stephen23]

"Haha a bit touchy about the eval function?"

Not really. I just read the documentation. Using eval forces beginners into writing slow, buggy, complex code. Why some beginners want to do this is a mystery to me, especially as much more efficient code is simpler to write!

[Krithika A]

To John: You might find this code and this link useful for specific directories:

   load(fullfile('..', 'folderName1', 'folderName2', 'filename.ext'))

https://uk.mathworks.com/help/matlab/matlab_env/specify-file-names.html

To Stephen: Well, they are beginners. That might give you a hint. It is useful to know about eval being slow. So far, I've only used really small sets of data with minimum processing so eval has been OK so far.

[Stephen23]

"Well, they are beginners."

Exactly. Which is why it is the right time to learn how to write good code.

"That might give you a hint."

A hint to what? I don't know what your "hint" is, and I cannot guess. Please tell me.

"It is useful to know about eval being slow"

Yes, it is. And it is also useful to know that dynamically accessing variable names makes code slow, complex, increases the risk of bugs, and makes code harder to debug. That is why the MATLAB documentation specifically recommends avoiding the code you gave in your answer.

"So far, I've only used really small sets of data with minimum processing so eval has been OK so far."

??? I don't see any reason why "beginners" means you have to teach them slow, complex, buggy code that is harder to debug. The simpler code in my answer is more efficient and easier to learn. Even beginners are allowed to read the documentation and learn how to write efficient code. It is actually easier!

[Krithika A]

Ok... This is going nowhere. My point is that beginners make mistakes because they are beginners. I agreed entirely with everything else you said. That was it. Goodbye for now.

[Stephen23]

"My point is that beginners make mistakes because they are beginners"

In reality all programmers make mistakes! Beginners make mistakes, experienced developers make mistakes, I make mistakes, you make mistakes, we all make mistakes. It is precisely because everyone makes mistakes that programmers have developed tools to help find mistakes in code, and to help fix them. These tools includes static code checking, which checks code in the MATLAB editor, before it is even run. These tools complete function names, show inputs and help, find syntax errors and suggest fixes for them:

https://www.mathworks.com/matlabcentral/answers/304528-tutorial-why-variables-should-not-be-named-dynamically-eval#answer_236123

But sadly some beginners like to use tools like eval, which mean that none of those code checking tools work. Those beginners force themselves into writing more complex code (the risk of bugs increases with code complexity) and at the same time they remove lots of useful tools that would actually help them to find and fix those bugs. This is one reason why experienced MATLAB users avoid eval, and instead write simpler, neater, less buggy, and easier-to-fix code. Your answer increases the risk of bugs and makes debugging harder: it is not clear to me how this is supposed to help those beginners who make mistakes (as you point out).