필터 지우기
필터 지우기

Tall arrays are not allowed to contain data of type struct

조회 수: 5 (최근 30일)
KAE
KAE 2021년 4월 22일
편집: KAE 2021년 4월 23일
I am having trouble with tall arrays in the following workflow,
  • Process data in a loop. In each iteration, generate structure containing results and save structure in a *.mat file
  • Read all *.mat files into a datastore
  • Turn datastore into a tall array. Gives the error "Tall arrays are not allowed to contain data of type struct"
  • Turn tall array into a timetable
Here's an example, which I generated by modifying Edric Ellis's answer here. Can you help me eliminate the error?
%% Setup
nRowLimit = 100; % Number of records in each file
nCalculation = 20; % Number of calculation iterations and also mat files
tStart = datetime(2013, 11, 1, 8, 0, 0); % For later use in generating datetime array
%% Prepare to save matlab files
iFileToSave = 0;
dirSave = 'c:\temp';
%%
for iCalc = 1:nCalculation % Loop through iterations of the calculation
clear StrucResults; % Structure to hold results of calculations
% Generate structure containing pretend calculation results
for iRow = 1:nRowLimit
% Generate example datetime to fill a datetime array
StrucResults.date(iRow) = tStart + 1000*iFileToSave + iRow;
% Generate example character to fill a cell array
StrucResults.label{iRow} = char(round(rand(1)*100));
% Generate example value to fill a numerical matrix
StrucResults.white(iRow, :) = rand(1, 5);
end
%% Save structure containing results to small mat file
iFileToSave = iFileToSave + 1;
% Make a distinct file name
fileSave = fullfile(dirSave, sprintf('FileContainingStruct_%05d.mat', iFileToSave));
save(fileSave, 'StrucResults');
end
%% Read the data back in as a tall array
% First create a datastore from all the mat files
fds = fileDatastore(fullfile(dirSave, '*.mat'), ... % Files to load
'ReadFcn', @(x)struct2table(load(x)), ... % Try to convert structure to table to avoid error in next code line
'UniformRead', true);
% Read into a tall array
tallData = tall(fds); % Throws error "Tall arrays are not allowed to contain data of type struct"
% Change to a tall timetable
tallTimeTable = table2timetable(tallData);
It's fine to eliminate the structure in favor of storing the results in individual variables, but I couldn't get that to work either.

채택된 답변

Edric Ellis
Edric Ellis 2021년 4월 23일
Your code was so nearly working. I made two tiny changes, marked with %###:
%% Setup
nRowLimit = 100; % Number of records in each file
nCalculation = 20; % Number of calculation iterations and also mat files
tStart = datetime(2013, 11, 1, 8, 0, 0); % For later use in generating datetime array
%% Prepare to save matlab files
iFileToSave = 0;
dirSave = fullfile(tempdir(), '809910');
if ~exist(dirSave, 'dir')
mkdir(dirSave);
end
%%
for iCalc = 1:nCalculation % Loop through iterations of the calculation
clear StrucResults; % Structure to hold results of calculations
% Generate structure containing pretend calculation results
%### Ensure vector fields are columns
for iRow = 1:nRowLimit
% Generate example datetime to fill a datetime array
StrucResults.date(iRow, 1) = tStart + 1000*iFileToSave + iRow;
% Generate example character to fill a cell array
StrucResults.label{iRow, 1} = char(round(rand(1)*100));
% Generate example value to fill a numerical matrix
StrucResults.white(iRow, :) = rand(1, 5);
end
%% Save structure containing results to small mat file
iFileToSave = iFileToSave + 1;
% Make a distinct file name
fileSave = fullfile(dirSave, sprintf('FileContainingStruct_%05d.mat', iFileToSave));
%### Use "save -struct" to ensure the fields of StrucResults are saved into the file directly.
save(fileSave, '-struct', 'StrucResults');
end
%% Read the data back in as a tall array
% First create a datastore from all the mat files
fds = fileDatastore(fullfile(dirSave, '*.mat'), ... % Files to load
'ReadFcn', @(x)struct2table(load(x)), ... % Try to convert structure to table to avoid error in next code line
'UniformRead', true);
% Read into a tall array
tallData = tall(fds); % Throws error "Tall arrays are not allowed to contain data of type struct"
% Change to a tall timetable
tallTimeTable = table2timetable(tallData);
% Check the results
head(tallTimeTable)
ans = 8×2 tall timetable date label white ____________________ _____ ________________________________________________________ 02-Nov-2013 08:00:00 {''} 0.24734 0.82558 0.41452 0.3178 0.41834 03-Nov-2013 08:00:00 {'B'} 0.49954 0.61133 0.76577 0.67823 0.50137 04-Nov-2013 08:00:00 {''} 0.099728 0.28918 0.42864 0.62252 0.61956 05-Nov-2013 08:00:00 {'H'} 0.67335 0.32173 0.25268 0.057868 0.17846 06-Nov-2013 08:00:00 {''} 0.82981 0.059187 0.34086 0.31046 0.25538 07-Nov-2013 08:00:00 {'@'} 0.53214 0.78883 0.30166 0.77258 0.68763 08-Nov-2013 08:00:00 {'Y'} 0.37458 0.19037 0.75301 0.62862 0.85332 09-Nov-2013 08:00:00 {'8'} 0.27491 0.22561 0.12215 0.46316 0.66067
The first change was simply to ensure that the vectors in the struct were columns.
The second more important change was to use save -struct to get rid of the extra layer of "struct" in the MAT file.

추가 답변 (0개)

카테고리

Help CenterFile Exchange에서 Large Files and Big Data에 대해 자세히 알아보기

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by