Problems with empty cell in a large matrix

Hello:

I have a large matrix of 102730 rows in the form of text file (sample text file is attached) with some header files in it. The first column show year, the next the month, followed by the day, and value1, value2 and value 3. Some of the cells are missing/empty. I want to fill these empty cells with NaN, so that they don't interefere with the next value. I am also attaching the text files of sample input (sample.txt) and desired output (Output.txt) for your reference.

Could anyone please let me know how to retrieve data?

Answer 1

[Peter Perkins]

You have a fixed-width file, and readtable needs to be told how it's organized, using fixedWidthImportOptions:

>> opts = fixedWidthImportOptions('NumVariables',7,...
                                  'DataLines',2,...
                                  'VariableNames',["Ind" "Mo" "dy" "V1" "V2" "AW" "X"],...
                                  'VariableWidths',[5 3 4 6 5 3 5],...
                                  'VariableTypes',repmat("double",1,7));
>> T = readtable('sample.txt',opts)
T =
  17×7 table
    Ind     Mo    dy     V1      V2     AW      X  
    ____    __    __    ____    ____    ___    ____
    1974    6     28    39.2    25.6      3       0
    1974    6     29    41.2    30.5      5       0
    1974    6     30    40.3    31.2      7       0
    1974    7      1    41.3    31.5     12       0
    1974    7      2    43.3    31.3     20       0
    1974    7      3    41.2     NaN     16    41.6
    1974    7      4    34.3    21.4     14    54.5
    1974    7      5    33.1    23.8      5       0
    1974    7      6    36.2    28.9      6       0
    1975    4     18    36.6    20.8     12       0
    1975    4     19    37.4    21.1      5       0
    1975    4     20    39.9      27      7       0
    1975    4     21    39.5    27.3      9       0
    1975    4     22     NaN     NaN    NaN     NaN
    1975    4     23    39.5    27.1      8       0
    1975    4     24    37.7      26     10       0
    1975    4     25    38.7    27.2     15       0

[Poulomi Ganguli]

I am using MATLAB 2017a. It says:

Undefined function or variable 'fixedWidthImportOptions'.

[madhan ravi]

+1

[Poulomi Ganguli]

Hi Peter:

I am getting this option is not available. So, I am unable to check this out.

[Peter Perkins]

Even though fixedWidthImportOptions was introduced in R2018b, you can try using detectImportOptions with file type 'fixedwidth'. Introduced in R2017a.

Answer 2

[KSSV]

Read about readtable. It will insert NaN's where ver values are missing.

T = readtable('sample.txt') ;

[madhan ravi]

Beware, the value next to 41.2 should be NaN whereas this answer gives 16. I have no idea why that’s happening.

[Peter Perkins]

That happens because readtable treats the file as space-delimited, when in fact it is fixed-width format. See my response below.

[madhan ravi]

Thank you Peter for the response :)

Answer 3

[Poulomi Ganguli]

Same here. If there are changes in number of digits before decimal, its not working. For example, "sample2.txt"

[madhan ravi]

Probably you would have to wait for others to shed some light here.