How do I calculate yearly average value for large data set with uneven time spacing?

I want to calculate yearly average value of 'd18O PDB' from the following data set (1984-1707):

<ftp://ftp.ncdc.noaa.gov/pub/data/paleo/coral/east_pacific/secas-10yr-iso.txt>

I have downloaded the text file into 2723 times 2 array and changed ('floored') the year to a whole number but I don't know how to go from there. Since the time spacing is uneven, it is hard to figure out how to solve the average value efficiently.

Answer 1

[Star Strider]

I couldn’t get the ftp call to work (I would not object to your sharing that since you obviously did), so I just copied the text file.

fidi = fopen('secas-10yr-iso.txt', 'rt');
Data = textscan(fidi, '%f%f%f%f', 'HeaderLines',4, 'CollectOutput',1);
fclose(fidi);
data = cell2mat(Data);
d18O_PDB = data(:,4);
Date_Data = [data(:,2) data(:,4)];
Y1 = Date_Data(end,1);                                          % Start Year
years = floor(Date_Data(:,1)-Date_Data(end,1))+1;               % Create Indices By Year
Yu = unique(years);                                             % Unique Year Indices
mean_d18O_PDB = accumarray(years, Date_Data(:,2), [], @mean);   % Calculate Annual Means
Output = flipud([Yu+floor(Y1)  mean_d18O_PDB]);                 % Matrix Of Means By Year

The flipud call is necessary because the first year is the first index, and flipping the matrix puts the years in the order they are in the original file.

EDIT: Added the ‘Date_Data’ (2723 x 2) array to correspond with the array in the Question.

[Tats the Tasmania]

Thanks so much. I have modified yours slightly on lines 8 and 11 so that the last row of Output becomes (1707, -5.78) instead of (1708, -5.4471). It seems to work well.

fidi = fopen('secas-10yr-iso.txt', 'rt');
Data = textscan(fidi, '%f%f%f%f', 'HeaderLines',4, 'CollectOutput',1);
fclose(fidi);
data = cell2mat(Data);
d18O_PDB = data(:,4);
Date_Data = [data(:,2) data(:,4)];
Y1 = Date_Data(end,1);                                          % Start Year
years = floor(Date_Data(:,1))-floor((Date_Data(end,1)))+1;      % Create Indices By Year
Yu = unique(years);                                             % Unique Year Indices
mean_d18O_PDB = accumarray(years, Date_Data(:,2), [], @mean);   % Calculate Annual Means
Output2 = flipud([Yu+floor(Y1)-1  mean_d18O_PDB]);              % Matrix Of Means By Year

[Star Strider]

My pleasure!

Answer 2

[David Young]

If your 2-column matrix is A, try:

years = floor(A(:,1));
data = A(:,2);
means = accumarray(years, data) ./ accumarray(years, ones(size(data)));

This should give you an array of means, indexed by year, with NaN in those years for which there is no data.

[Tats the Tasmania]

Thanks David. It worked. All I had to do next was flip everything around and remove all the NaNs.