Import CSV from nth row. n is variable across files

Hi,

How do I generalize importing a csv file. The csv file contains lot of information before the data headers and the actual data starts. For example for one CSV file the headers ( variablesnames) start at 100th row and row 101 is data. It might not be the same for another csv file. Also the data ends at a certain row, but in the first column there is addtional row which has end of file (EOF) and the all the others columns are empty.

Whenever I use

readtable

function, it treats the headers as NaN.

Also, the common header between all the CSVs that I am using is 'time' which might be 100th row in one csv file but be in another row in another csv file.

The first column is time stamps but I can treat that as a data point so it is easier for me to plot stacked plot.

Thanks !

Bhargav

[Stephen23]

Please upload a sample data file by clicking the paperclip button.

[Bhargav]

I did figure out how to import from a certain row. Now next thing I want to do is just import certain columns. the output is not able to give active channels. It is giving me all the columns with resistance and the column next to it

function data = readcathimpdata(filename)
% Read the CSV file
data = readtable(filename);

% Remove the last row (EOF)
data(end, :) = [];

% Initialize variable for active channels
active_channels = [];

% Find the row index where 'channel' appears
channel_row = find(strcmp(data{:,1}, 'channel'), 1, 'first');

% If 'channel' row is found
if ~isempty(channel_row)
    % Find the first empty cell after 'channel' row
    empty_cell_row = find(cellfun(@isempty, data{channel_row+1:end, 1}), 1, 'first') + channel_row;
    
    % Extract active channels between 'channel' row and the first empty cell after it
    if ~isempty(empty_cell_row)
        active_channels = data{channel_row+1:empty_cell_row-1, 1};
    else
        active_channels = data{channel_row+1:end, 1};
    end
    
    % Print active channels
    if ~isempty(active_channels)
        disp('Active Channels:');
        disp(active_channels);
    else
        disp('No active channels found.');
    end
else
    disp('Row with "channel" not found.');
    disp('No active channels found.');
end

% Extract data headers
data_headers = data.Properties.VariableNames;

% Find columns containing 'RESISTANCE'
resist_cols = contains(data_headers, 'RESISTANCE');

% Initialize a cell array to store relevant column indices
relevant_cols_indices = [];

% Find the column indices of 'RESISTANCE' columns
resistance_indices = find(resist_cols);

% Extract the column indices of 't_dws'
t_dws_index = find(strcmp(data_headers, 't_dws'));

% Add 't_dws' index to relevant columns indices
relevant_cols_indices = [relevant_cols_indices, t_dws_index];

% Extract columns next to each 'RESISTANCE' column
for idx = resistance_indices
    if idx < numel(data_headers)
        relevant_cols_indices = [relevant_cols_indices, idx, idx+1];
    end
end

% Sort the relevant column indices
relevant_cols_indices = sort(relevant_cols_indices);

% Extract data from relevant columns
data = data(:, relevant_cols_indices);
end

[Alexander]

Is this an answer or the update of your question? If it's an update move it to a comment.

[Bhargav]

No , its not an answer to the question. this is the follow up question.

[Bhargav]

Should I create a new thread ?

[Alexander]

No, everything is OK. @Voss has moved your answer to a comment.

Answer 1

[Alexander]

I think (the specialists here might correct me) you don't get a ready to use function, that solves your problem. Here a suggestion:

% https://de.mathworks.com/matlabcentral/answers/2080816-import-csv-from-nth-row-n-is-variable-across-files

datlength = 60000; % Adjust it to your needs

Data = char(zeros(1,datlength));

fid = fopen('example_csv.csv');

while (feof(fid) == 0)

dyData = pad(fgetl(fid),datlength);

Data = [Data; dyData(1:datlength)];

end

fclose(fid);

Data=deblank(Data); % Reduce Data to the max.

% Now you have your csv in "Data". You can parse it to your needs line by

% line.

If I get some more information I'll try to help further on. But not today (UTC+2 ;-).