"ReadTable" column reading error/bug?

Hello,

I am attempting to read this csv file with the 'readTable' function, and when I do the columns of the original csv file are not read into matlab correctly. Instead, they are jumbled up, and some columns appear when they never existed in the original csv file (see attached csv file). I have tried this with another csv file created from the same program, and it works without issue.

filepath = '/Users/Nick/Desktop/SLO/CellProfiler/3kDa_PC3_P19_5x/Attept2Pipeline_Perinuclear_Ring';
T = readtable(filepath);

I get the warning "column headers were modified" but even if I were to do:

readtable(filepath,,'ReadVariableNames',false)

(or read variable names, true) I still get the same exact column mess up.

Is there something I am doing wrong here?

Thank you for your time!

[Nicholas Scott]

For completeness, this is the other csv file that was read properly. Both have underscores, so matlab shouldn't be splitting the columns based on '_' as a separator. I am very confused!

Answer 1

[Stephen23]

Simpler and more efficient:

T = readtable('Attept2Pipeline_Perinuclear_Ring.csv', 'Delimiter',',')

T = 6826×9 table
    ImageNumber    ObjectNumber                                FileName_Alexa488                                                            FileName_BrightField                                                           FileName_PIStained                                   PathName_Alexa488        PathName_BrightField        PathName_PIStained       Intensity_MeanIntensity_Alexa488
    ___________    ____________    __________________________________________________________________________    __________________________________________________________________________    __________________________________________________________________________    _______________________    _______________________    _______________________    ________________________________

         1               1         {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}               0.0037245            
         1               2         {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}               0.0030013            
         1               3         {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}               0.0033284            
         1               4         {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}               0.0035036            
         1               5         {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}               0.0033021            
         1               6         {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}               0.0037373            
         1               7         {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}               0.0038118            
         1               8         {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}               0.0038424            
         1               9         {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}               0.0039416            
         1              10         {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}               0.0033224            
         1              11         {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}               0.0035364            
         1              12         {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}               0.0037767            
         1              13         {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}               0.0035306            
         1              14         {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}               0.0034484            
         1              15         {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}               0.0031682            
         1              16         {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved_PC3.tif'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}               0.0029068            

T = readtable('Attept2Pipeline_Perinuclear_Ring_SuccessfulRead.csv', 'Delimiter',',')

T = 6826×9 table
    ImageNumber    ObjectNumber                                            FileName_Alexa488                                                                                  FileName_BrightField                                                                                  FileName_PIStained                                               PathName_Alexa488        PathName_BrightField        PathName_PIStained       Intensity_MeanIntensity_Alexa488
    ___________    ____________    _________________________________________________________________________________________________    _________________________________________________________________________________________________    _________________________________________________________________________________________________    _______________________    _______________________    _______________________    ________________________________

         1               1         {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}               0.0037245            
         1               2         {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}               0.0030013            
         1               3         {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}               0.0033284            
         1               4         {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}               0.0035036            
         1               5         {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}               0.0033021            
         1               6         {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}               0.0037373            
         1               7         {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}               0.0038118            
         1               8         {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}               0.0038424            
         1               9         {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}               0.0039416            
         1              10         {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}               0.0033224            
         1              11         {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}               0.0035364            
         1              12         {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}               0.0037767            
         1              13         {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}               0.0035306            
         1              14         {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}               0.0034484            
         1              15         {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}               0.0031682            
         1              16         {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'3kDa_P19_ZDC_5xConcentrationAlexProtocol_take2MoreTimeResolved.oir Group:1 Level:1 Area:1.tif'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}    {'/Users/Nick/Desktop'}               0.0029068            

[Voss]

It took me a few minutes looking through the documentation for readtable(), but I eventually found it.

[Nicholas Scott]

Excellent! Thank you both so much for your help! I find it odd that it read two similar csv files differently, and specifying the delimiter as a ',' in a csv, but if it works it works!

[Stephen23]

"I find it odd that it read two similar csv files differently..."

It is not very odd: READTABLE (and family) parses the file text and performs a whole lot of analysis of the file content to automagically determine things like the delimiter character, the field types, etc. etc. How do you think it knows that the first column is numeric? How does it even know what constitutes the first column?

This file parsing works in many cases, but of course any algorithm can get confused. The more the file deviates from a basic format with "standard" characters, the more likely it is that the file parsing will algorithm will need some hints to work correctly, which is exactly why this exists too:

https://www.mathworks.com/help/matlab/ref/detectimportoptions.html

I suspect that the filenames (in the CSV files) are the problem: they contain many underscores, whitespace, and periods which are just primed to confuse poor READTABLE's algorithm.

[Nicholas Scott]

I see. Thank you for clarifying for me! I also like that little comedy you put in there. :) "automagically"