How to find selection frequency in a txt file?

Hello all, I have 10 txt files (I share an example) and each file has a list of selected items. I have 13 items in total. What I want to know is how many times each item was selected but I am getting an error (Input must be a MAT-file or an ASCII file containing numeric data with same number of columns in each row) message. I think the problem is the varying length of each line. How can I fix it? Thanks for the help.

txtFiles = dir('*.txt'); 
nFiles = length(txtFiles);        

for i = 1:nFiles
    txtCnt = load(txtFiles(i).name);            
    fprintf('-----%s-----\n',txtFiles(i).name);
        
    Frq = zeros(1,13);
    for Indx = 1:13
        Frq(Indx) = sum(txtCnt == Indx);
    end

    Results = [1:13; Frq]
    idx = find(Frq > mean(Frq))
end

Error using load
Unable to read file 'Testing.txt'. Input must be a MAT-file or an ASCII file containing numeric data with same number of columns in each row.

Answer 1

[Cris LaPierre]

Do not use load. I would probably use readmatrix. Note that your rows do not all have the same number of elements. These rows are padded with NaN.

data = readmatrix("Testing.txt")

data = 422×5

2 6 8 NaN NaN 2 6 8 13 NaN 2 6 8 12 NaN 6 8 NaN NaN NaN 6 8 12 NaN NaN 6 8 11 NaN NaN 6 8 13 NaN NaN 1 6 8 NaN NaN 1 6 8 13 NaN 3 6 8 12 NaN

[MByk]

Yes sir they do not have the same number of elements. Using fgetl might solve the NaN problem? I can read the file line by line and store each line in an array. But if i didn’t get wrong fgetl will return character array therefore I need to convert them into numeric.

[Cris LaPierre]

There has to be a value there. If you don't want it to be NaN, you can use fillmissing to replace NaN with something else. However, NaN is a perfectly acceptable value.

[Cris LaPierre]

I suggest also using the 'all' option when summing your array. This will return a scalar rather than a vector, which can then be stored in Frq without causing an error.

S = sum(A,"all")

txtFiles = dir('*.txt'); 
nFiles = length(txtFiles);        

for i = 1:nFiles
    txtCnt = readmatrix(txtFiles(i).name);            
    fprintf('-----%s-----\n',txtFiles(i).name);
        
    Frq = zeros(1,13);
    for Indx = 1:13
        Frq(Indx) = sum(txtCnt == Indx,'all');
    end

    Results = [1:13; Frq]
    idx = find(Frq > mean(Frq))
end

-----Testing.txt-----

Results = 2×13

1 2 3 4 5 6 7 8 9 10 11 12 13 86 90 105 103 110 121 89 68 115 97 100 100 85

idx = 1×7

3 4 5 6 9 11 12

[MByk]

Thank you for your help sir. The best way is what you suggest, it worked like a charm.

[MByk]

Sorry to bother you again but I found a mistake in the results of this file. What could be the problem? It takes half of the file.

Results = 6 0 0 0 2 0 4 0 0 6 0 0 5.

It must be: 8 0 0 0 6 0 7 0 0 8 0 0 6.

EDIT: Adding code and running here

txtFiles = dir('*.txt'); 
nFiles = length(txtFiles);        

for i = 1:nFiles
    txtCnt = readmatrix(txtFiles(i).name)            
    fprintf('-----%s-----\n',txtFiles(i).name);
        
    Frq = zeros(1,13);
    for Indx = 1:13
        Frq(Indx) = sum(txtCnt == Indx,'all');
    end

    Results = [1:13; Frq]
    idx = find(Frq > mean(Frq))
end

txtCnt = 9×3

1 5 10 1 5 13 7 10 13 10 13 NaN 7 10 NaN 1 7 13 1 10 13 1 10 NaN 1 7 NaN

-----Error.txt-----

Results = 2×13

1 2 3 4 5 6 7 8 9 10 11 12 13 6 0 0 0 2 0 4 0 0 6 0 0 5

idx = 1×5

1 5 7 10 13

[Cris LaPierre]

Looks like MATLAB is trying to guess where the data starts. This is a result of having a different number of values in each row. Use the 'NumHeaderLines' name value pair to indicate that the data starts on the first line.

txtFiles = dir('*.txt'); 
nFiles = length(txtFiles);        

for i = 1:nFiles
    txtCnt = readmatrix(txtFiles(i).name,'NumHeaderLines',0)
    fprintf('-----%s-----\n',txtFiles(i).name);
        
    Frq = zeros(1,13);
    for Indx = 1:13
        Frq(Indx) = sum(txtCnt == Indx,'all');
    end

    Results = [1:13; Frq]
    idx = find(Frq > mean(Frq))
end

txtCnt = 13×4

5 7 13 NaN 5 10 NaN NaN 1 5 7 NaN 1 5 7 10 1 5 10 NaN 1 5 13 NaN 7 10 13 NaN 10 13 NaN NaN 7 10 NaN NaN 1 7 13 NaN

-----Error.txt-----

Results = 2×13

1 2 3 4 5 6 7 8 9 10 11 12 13 8 0 0 0 6 0 7 0 0 8 0 0 6

idx = 1×5

1 5 7 10 13

[MByk]

I spent hours trying to find hidden/special characters that could cause the error. :) I also tested xls format that resulted in same error. Thank you again.