Duplicate table variable name: 'VarName2'.

I have 15 minute interval data of power produced on a daily basis from nine different sources (coal, gas, nuclear etc.) for the month of Jan 2012. I want data in a single column for each source, meaning 9 different columns since I have 9 sources. Currently it is a row wise data for each source for each day of the month. I am using the following code and getting an error (Duplicate table variable name: 'VarName2'.) :

ndata = data;
ndata(:,1)=[];
id = eye(9);
id = repmat(id,31,1);   % 31 day
output = [];
for i = 1:9;
    one = id(:,i);
    one = logical(one);
    temp = ndata(one,:);
    output = [output temp];
end

Duplicate table variable name: 'VarName2'.

[Image Analyst]

I don't see you creating a table in that code. There is no call to the table() function. Nor do I see you calling the "VarName2" field of the table.

Please post the code that threw the error - that means ALL the red text, including the line number, line of code, error message - everything.

[Kushal Bhalla]

ndata = data;
ndata(:,1)=[];
id = eye(9);
id = repmat(id,31,1);   % 31 day
output = [];
for i = 1:9;
    one = id(:,i);
    one = logical(one);
    temp = ndata(one,:);
    output = [output temp];
end
Error using untitled (line 10)
Duplicate table variable name: 'VarName2'.

[Scott MacKenzie]

I don't see anything wrong in line 10. Of course, the 10th line in the excerpt of code in your question is not the 10th line in the code you are executing. You might consider posting your data (or a subset of the data) and code that can be executed to reproduce the error.

[Kushal Bhalla]

ndata = data;
ndata(:,1)=[];
id = eye(9);
id = repmat(id,31,1);   % 31 day
output = [];
for i = 1:9;
    one = id(:,i);
    one = logical(one);
    temp = ndata(one,:);
    output = [output temp];
end

[Scott MacKenzie]

The data portion of the table forms a 279x96 matrix. The columns are 96 measurements on 15-minute intervals. The rows correspond to the 9 sources, repeated 31 times. You are trying to organized the data into 9 columns. Are you looking for 31x96=2976 rows? Please explain.

[Kushal Bhalla]

Sorry Sir for the inordinate delay in replying. I was reeling under a bout of illness. Had symptoms akin to COVID, but now feeling better.

Sir, you are right, I am trying to have a 2976 * 9 matrix. The columns will represent the 9 sources of power production. The rows will showcase the power produced from each source at a fifteen minute interval for the month of Jan 2012.

I was using the following code which generated an error in the for loop statement as previously shared with you.

ndata = data;
ndata(:,1)=[];
id = eye(9);
id = repmat(id,31,1);   % 31 day
output = [];
for i = 1:9;
    one = id(:,i);
    one = logical(one);
    temp = ndata(one,:);
    output = [output temp];
end
d = 96;
biomass = output(:,1:d);
coal = output(:,d+1:d*2);
gas= output(:,d*2+1:d*3);
gas2 = output(:,d*3+1:d*4);
hydro = output(:,d*4+1:d*5);
nuclear = output(:,d*5+1:d*6);
oth = output(:,d*6+1:d*7);
sun = output (:,d*7+1:d*8);
wnd = output(:,d*8+1:d*9);
biomass = biomass';
[m,n] = size(biomass);
coal = coal';
gas = gas';
gas2 = gas2';
hydro = hydro';
nuclear = nuclear';
oth = oth';
sun = sun';
wnd = wnd';

biomass1201 = reshape (biomass,m*n,1)
coal1201 =reshape(coal,m*n,1);
gas1201 = reshape(gas,m*n,1);
gas21201 = reshape(gas2,m*n,1);
hydro1201 = reshape(hydro,m*n,1);
nuclear1201 = reshape(nuclear,m*n,1);
oth1201 = reshape(oth,m*n,1);
sun1201 = reshape(sun,m*n,1)
wnd1201 = reshape(wnd,m*n,1);
save data1201 biomass1201 coal1201 gas1201 gas21201 hydro1201 nuclear1201 oth1201 sun1201 wnd1201;

[David Ebert]

For the loop, try one easy addition:

Add a ";" in output = [output temp]; so it looks like:

output = [output ; temp];

for i = 1:9;

one = id(:,i);

one = logical(one);

temp = ndata(one,:);

output = [output temp];

end

Cheers,

David

Answer 1

[Walter Roberson]

Your code cannot work.

You have a table() object that includes a non-numeric first column and several non-numeric trailing columns (the several commas in a row will become non-numeric variables).

You delete the first (non-numeric) variable from the table... leaving the empty non-numeric ones at the end of the list.

You go through and extract various rows from the table. You use () indexing so each result is a table object.

You try to [] the extracted table objects, but that fails because the table objects all have the same variable name but you cannot have duplicate variables in the same table.

If you were to use {} indexing instead of () then that would fail because you can only extract multiple variables if they all have the same data type, but the trailing (empty) columns are a different data type.

You could get rid of trailing empty columns and then use {} indexing, or you could use vertical concatenation instead of horizontal, or you could switch to cell array or you could put each extract as a distinct variable in the [] table.

[Kushal Bhalla]

Sorry Sir, for the late reply. I had suffered a bout of illness. Now convalescing.

Sir, being a novice, I am struggling to understand the root cause of the problem as enunciated by you.

1) Non numeric trailing columns?

2) table object with the same variable name?

3) trailing empty colums?

4) putting each extract as a distinct variable?

The full code is pasted below. I get error in the for loop. Earlier this code used to run fine.

ndata = data;
ndata(:,1)=[];
id = eye(9);
id = repmat(id,31,1);   % 31 day
output = [];
for i = 1:9;
    one = id(:,i);
    one = logical(one);
    temp = ndata(one,:);
    output = [output temp];
end
d = 96;
biomass = output(:,1:d);
coal = output(:,d+1:d*2);
gas= output(:,d*2+1:d*3);
gas2 = output(:,d*3+1:d*4);
hydro = output(:,d*4+1:d*5);
nuclear = output(:,d*5+1:d*6);
oth = output(:,d*6+1:d*7);
sun = output (:,d*7+1:d*8);
wnd = output(:,d*8+1:d*9);
biomass = biomass';
[m,n] = size(biomass);
coal = coal';
gas = gas';
gas2 = gas2';
hydro = hydro';
nuclear = nuclear';
oth = oth';
sun = sun';
wnd = wnd';

biomass1201 = reshape (biomass,m*n,1)
coal1201 =reshape(coal,m*n,1);
gas1201 = reshape(gas,m*n,1);
gas21201 = reshape(gas2,m*n,1);
hydro1201 = reshape(hydro,m*n,1);
nuclear1201 = reshape(nuclear,m*n,1);
oth1201 = reshape(oth,m*n,1);
sun1201 = reshape(sun,m*n,1)
wnd1201 = reshape(wnd,m*n,1);
save data1201 biomass1201 coal1201 gas1201 gas21201 hydro1201 nuclear1201 oth1201 sun1201 wnd1201;

[Walter Roberson]

Look at line 2 of your input. It is

01/01/12-Biomass,8.30,7.61,7.32,7.32,7.32,7.43,7.83,7.98,8.19,7.97,7.31,7.63,7.94,8.04,8.23,8.13,8.14,8.14,8.08,7.78,7.79,7.80,7.78,7.80,7.86,7.86,7.88,7.88,7.87,7.87,7.84,7.87,7.86,7.72,6.92,6.96,6.96,6.95,6.95,6.96,6.92,6.69,6.55,6.92,7.76,7.91,7.90,7.92,7.90,7.95,8.02,8.21,8.35,8.36,8.35,8.37,8.35,8.36,8.27,8.34,8.34,8.35,7.76,7.70,7.74,7.77,8.03,8.14,8.07,8.08,8.08,7.90,7.77,7.79,7.81,7.82,7.90,8.07,8.23,8.30,8.39,8.39,8.39,8.39,8.38,8.39,8.35,8.31,8.29,8.18,8.16,8.15,8.16,8.15,8.15,8.16,,,,

Notice the 4 commas at the end: those indicate that you have four trailing columns that have no value for that row. readtable() will decide that those columns are not numeric. Your first column is also not numeric.

Your code deletes the first (non-numeric) entry from the table. The resulting table() object will not have only numeric entries.

ndata will be a table() object with variable names starting from 'Var2' .

id = eye(9);
id = repmat(id,31,1);   % 31 day

Those all ones and zeros.

for i = 1:9;
    one = id(:,i);

ones and zeros.

    one = logical(one);

transformed into true and false.

    temp = ndata(one,:);

and used as a column mask.

You are using round-bracket indexing, () with a table() object, so the result is a table, with variable names starting from 'Var2' -- the same variable names as in the main ndata table.

    output = [output temp];

and there you try to put this new table beside the accumulated table. This will work on the first round when output is empty, but on the second round, output is already a table with variable names starting from 'Var2', and you are now trying to put beside it a table that already has variable names starting from 'Var2'. That would be an attempt to have two different variables named 'Var2' in the same table, and that will fail.

How can you fix it? It is not difficult: change

ndata = data;
ndata(:,1)=[];

to

ndata = data{:,2:97};

The result will be a numeric array instead of a table() object, and then when you extract portions of it, the result will be numeric.

[Kushal Bhalla]

Thank you Sir for clarifying the problem. Also thanks for suggesting a way out to run the original code. Appreciate!

Answer 2

[Scott MacKenzie]

Here are three solutions. They all give nine columns, one for each source, as per your question. The first table (dataNew1) is a simple reorganization with 31 x 96 = 2976 rows. The second table (dataNew2) aggregates the data by computing the mean value over 31 days in the month. This yields 96 rows, one for each 15-minute interval. The third table (dataNew3) aggregates the data by computing the mean over the 15-minute intervals. This yields 31 rows, one for each day.

data = readtable('https://www.mathworks.com/matlabcentral/answers/uploaded_files/612335/data.csv');
sourceNames = data{1:9,1};
sourceNames = split(sourceNames, '-'); % date + source name
sourceNames = sourceNames(:,2); % just the source name

% reorganize the data (show each step in a separate variable)
d1 = data{:,2:end};
d2 = d1(:);

% organize data into 9 columns and 31x96 = 2976 rows
d3 = reshape(d2,9,[]);
d4 = d3';
dataNew1 = array2table(d4); % 2976x9
dataNew1.Properties.VariableNames = sourceNames;

% organize data into 9 columns and 96 rows
d5 = reshape(d2,9,31,96);
d6 = mean(d5,2); % compute mean over days (31)
d7 = squeeze(d6);
d8 = d7';
dataNew2 = array2table(d8); % 96x9
dataNew2.Properties.VariableNames = sourceNames;

% organize the data into 9 columns and 31 rows
d9 = reshape(d2,9,31,96);
d10 = mean(d9,3); % compute mean over 15-minute intervals (96)
d11 = d10';
dataNew3 = array2table(d11); % 31x9
dataNew3.Properties.VariableNames = sourceNames;

[Kushal Bhalla]

Sir,

Among the three solutions prescribed by you, I had sought after the first one. However, there is a discrepancy. In the end result of solution one (dataNew1), the sequence of observations in the rows need to depict the 15 minute interval starting from the first day of the month. Meaning, the first 96 rows for all the 9 columns should be for the day 1 of the month. The next 96 rows for the second day and so on. That will make it an ideal time series data of power produced from different sources at a 15 minute interval. Please suggest a remedy for this.

Moreover, could you also suggest a way for aggregating the values for four fifteen minute intervals in the original data, so I may construct a new hourly data. This new hourly data should be a 744 * 9 matrix. This should provide an ideal time series data of power produced from different sources at an hourly interval.

I really appreciate your intellectual wisdom in offering solution 2 and 3. That renders another perspective on observing the daily means (mean of 96 values) and means for each 15 minute interval for different days of the month.

Thanks a lot Sir.

[Scott MacKenzie]

I believe your original question, as posed, has been answered. If new issues emerge, may I suggest your organize these into a new question. Good luck.

[Kushal Bhalla]

Sure Sir.

Thank you so much for your prompt help and assistance. Really appreciate!