Read large number of .h5 datasets

조회 수: 1 (최근 30일)
Ashraf El Gaaly
Ashraf El Gaaly 2018년 8월 15일
댓글: per isakson 2018년 10월 28일
Hi all,
I'm working with these h5 files that have tens of thousands of datasets, that contains vectors of numerical values and all of the same size. My goal is to read the datasets and create one large matrix from these vectors. The datasets are named 0-xxxxx I was able to read them and get the matrix but it takes forever to do so. I was wondering if you can take a look at my code and suggest a way to make it run faster
t =[];
for i = 0:40400 % there are 40401 datasets in this particular file
j = int2str(i);
p = '/mesh/'; % The parent group
s = strcat(p,j); % to create the full path of a dataset e.g. '/mesh/0'
r = h5read('temp.h5',s); % the file name is temp and s has the dataset path
t = [t;r];
end
in this file there are 40401 datasets, each has 80802x1 vector of numerical values. Therefore eventually I want to create 80802x40401 matrix. This code has been running for over 6 hours now and still not done. I tried another smaller file yesterday with 10801 datasets and it took about 4 hours to get the answer.
  댓글 수: 3
Ashraf El Gaaly
Ashraf El Gaaly 2018년 10월 24일
Yes I have enough ram. The program takes over a day to run, but eventually runs
per isakson
per isakson 2018년 10월 28일
Did you read and understand the text on Preallocation?

댓글을 달려면 로그인하십시오.

답변 (0개)

카테고리

Help CenterFile Exchange에서 HDF5에 대해 자세히 알아보기

태그

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by