Error when reading a csv using csvread

I am trying to read in a .csv file, which I have read in previously with no trouble.

Here are my command and errors:

>> A=csvread('Tawharanui5278_PSD_1sHammingWindow_50%Overlap_output.csv');

Error using dlmread (line 147)
Mismatch between file and format character vector.
Trouble reading 'Numeric' field from file (row number 394, field number 62257) ==>
                                                                                                                                                                                                        ...
Error in csvread (line 48)
    m=dlmread(filename, ',', r, c); 
    

Any ideas what the problem is? I think the error suggests the problem is with the command rather than the file?

Thanks for your help.

Answer 1

[Walter Roberson]

Row number 394 of the file is missing field number 62257

You should switch to using readtable() or readmatrix()

[Louise Wilson]

Thanks for your very prompt answer Walter! I have the files I am trying to read in several places and I've found csvread works when the files are in my external HD but not when they're on my shared drive. Do you know why this would be? About to try readmatrix().

[Walter Roberson]

That does sound a bit odd. What kind of shared drive is it? Is it Microsoft OneDrive ?

[Louise Wilson]

Just tried readmatrix and I get the following error:

>> A=readmatrix('Noises5281_PSD_1sHammingWindow_50%Overlap_output.csv');
Failed to convert character code.

[Louise Wilson]

The drive is a University Homeshare drive on my desktop, provided by my school, not sure if that helps?

[Walter Roberson]

That is odd. Would it be possible to zip the file and attach it for us to have a look at? There is a 5 megabyte limit for that, though, and I am not sure it will zip small enough.

[Louise Wilson]

Unfortunately the file is 7GB! It works perfectly when I use the file in the original location where it was created (with Matlab). But it doesn't work on copies of the file. How should I back up such large files? The file looks to be copied and is the same size in the copied location but doesn't open. Any ideas?

[Walter Roberson]

Just to check the obvious: do you have 7 Gb of free space for the shared drive to copy to as cache, and maybe another 7 Gb as well if it thinks you are modifying the file ?

[Louise Wilson]

Should I perhaps change the encoding?

https://au.mathworks.com/help/simulink/slref/slcharacterencoding.html

[Walter Roberson]

I was not familiar with Homeshare; when I look around I see that various universities have programs under that name or under the name "Home Share" or under the name "U drive". I see that some of them back up automatically and some do not.

I would suggest a test: run an SHA1 hash on both versions of the file and see if they come out the same; https://www.microsoft.com/en-ca/download/details.aspx?id=11533

[Walter Roberson]

I suggest you try my detect_UTF_encoding utility that I have attached.

function [encoding, fileclass, file_description, bytes_per_char, BOM_size, byte2char, are_you_sure] = detect_UTF_encoding(FILENAME, allow_warning)
    %detect_UTF_encoding determines which UTF encoding has been used for a
    %file, and returns:
    % * an encoding name;
    % * 'no file' or 'file exists'
    % * text description of the kind of file it is
    % * number of bytes per character;
    % * size of the Byte Order Mark used in the file;
    % * a handle to a routine that converts byte vectors to characters taking into account
    %   byte swapping;
    % * a flag indicating certainty of that the output is correct
    % allow_warning can be set to false for automated use instead of
    % interactive use

[Louise Wilson]

Hi Walter, thanks for the code. Before I tried it, I re-copied and pasted my files using TERACOPY! This has fixed the problem so must have been bad ctrl+c and ctrl+p problem...