CSV file import in Matlab

Hello everybody,

I have to import data from multiple csv files (more than 4000). Each csv file contains 7 columns, between 1 and 6 rows and 1 header-line. It contains multiple types of data, this format : formatSpec = '%f %f %f %s %f %f %f'. I used this to extract the data :

%%How many file ?
files = dir('C:\Users\titin\Documents\MATLAB\UALG\RADAR07\*csv');
num_files = length(files);
%%Extract data from files
for i=1:num_files
for j=1:1
% open files
fid = fopen(files(i).name
% read data from csv file
readData = textscan(fid,'%f %f %f %s %f %f %f','HeaderLines',1,'Delimiter',',','TextType','string','ReturnOnError',false,'EndOfLine','\r\n');
% extract data from readData
    UserID(i,j) = readData(1,1);
    Longitude(i,j) = readData(1,2);
    Latitude(i,j) = readData(1,3);
    Timestamp(i,j) = readData(1,4);
    SpeedOverGround(i,j) = readData(1,5);
    NavigationStatus(i,j) = readData(1,6);
    ShipType(i,j) = readData(1,7);
end

end

The problem is, when I have a csv file which have more than 1 rows it creates a cell into an another cell like this :

For the numeric cell its not a problem because I used the function : cell2mat, and each cell was divided. But for the string cell, I don't know how to proceed to not have cells into cells. I tried to use the function cellstr but when there is a cell into cell I have an error.

If someone can help it would be great !

Thanks !

[Guillaume]

An example of file to be imported would be very useful to help solve your problem.

[Corentin Hurel]

Hello,

This is the type of file I have to import.

Regards

[Stephen23]

@Corentin Hurel: what Guillaume requested is an example file, not a screen-shot. A screen-shot cannot be imported, nor can we test code on a screen-shot.

You can upload an example file by clicking the paperclip button.

[Corentin Hurel]

Oh ok ,

here it is

[Guillaume]

We need an example file that shows the problem.

Any reason you're not using readtable to do the parsing for you?

Answer 1

[Guillaume]

Assuming you just want the multiple rows in the sub-cells to be converted into rows in the main array, and assuming that the main cell array is indeed a column cell vector (i.e. Nx1) and the subcells are also column cell vectors, then

if any(cellfun(@iscell, yourcellarray))
   yourcellarray = vertcat(yourcellarray{:});
end

But you'd be better off fixing the import, something we can only help with if we have an example of a file causing problem.

Even better would be to use readtable to do the parsing for you.

[Corentin Hurel]

Thanks for this answer, this was exactly what I needed!

Regards

Answer 2

[Ari]

The function textscan returns data into a string or cell array, and when you have multiple rows in your csv files it will return a cell array. To access the contents of a cell array you can use the { } operator. For example you can use the following in your code:

TimestampArray = readData(1,4);
TimestampStrings = TimestampArray{:};

Now TimestampStrings will be a Nx1 string array, you can access the elements using

ithTimestamp = TimestampStrings(i); % where i is an integer

For more details on indexing matrix and cell array, view the documentation here.

[Corentin Hurel]

Hello, Thanks for your answer!

I have some trouble to use the {}. In fact I use :

if true
  % code
for i=1:num_files_j1
for j=1:1
TimestampArray(i,j)=readData(1,4)
TimestampString(i,j)=TimestampArray{i,:}
end
end
end

But I have the following error : Assignment has more non-singleton rhs dimensions than non-singleton subscripts

Error in Version1_Day1 (line 19)

Time(i,j) = Timestamp{i,:};

Do you know why ?

Regards

[Guillaume]

Ari's answer is mostly accurate but has one major problem:

TimestampStrings = TimestampArray{:};

While that would work, it is very unlikely to produce the desired result. It will assign only the first element of the cell array to TimestampStrings. If that is the intent then it's extremely bad syntax as the right hand side is expanded into a comma separated list of which all but the first element is used.

TimestampStrings = TimestampArray{1};

would be a lot more correct.

TimestampString(i,j) = TimestampArray{i,:}

is never going to work. I'm not sure what the intent is, but you can't stuff the multiple values returned by the right hand side into the scalar value on the left hand side.

[Corentin Hurel]

Hello, Thanks for your answer !

I have a cell array named TimestampArray that contains 109 rows of string. In certains cell i have an another cell with multiple rows. I want to have just an array of string without cells into cells.

Do you know how to do this?

Regards

[Guillaume]

Well, what should happen with these multiple rows in these subcells? Convert them into rows into the main cell array?