Problem in reading H5 data compression or chunks (TIFF)

Dear all,

I have a problem that might comes from some compression issue or from reading the data chunks (image slices) of the hdf5 file.

I have a 2GB H5 files in which I want to extract a specific data set (it contains 2000 Tiff). It was written with h5py 3.6.0 version (supposed to support HDF5 versions 1.8.4 and higher). My Matlab is 1.8.12 >> [major,minor,relnum] = H5.get_libversion().

When I did :

>> h5disp(tmp_file,'/entry_0000/instrument/eiger/data')

HDF5 fresh_pinna_ind1_scan_100gr_e_00001_data_000000.h5

Dataset 'data'

Size: 2070x2167x2000

MaxSize: 2070x2167x2000

Datatype: H5T_STD_U32LE (uint32)

ChunkSize: 2070x2167x1

Filters: unrecognized filter (bitshuffle; see https://github.com/kiyo-masui/bitshuffle)

FillValue: 0

Attributes:

'interpretation': 'image'

But when I try to load the data :

>> rawdata = h5read(tmp_file,'/entry_0000/instrument/eiger/data',[1 1 1],[1 1 1])

Error using h5readc

The HDF5 library encountered an error and produced the following stack trace information:

H5PL__find can't open directory

H5PL_load search in paths failed

H5Z_pipeline required filter 'bitshuffle; see https://github.com/kiyo-masui/bitshuffle' is not registered

H5D__chunk_lock data pipeline read failed

H5D__chunk_read unable to read raw data chunk

H5D__read can't read data

H5Dread can't read data

Error in h5read (line 93)

[data,var_class] = h5readc(Filename,Dataset,start,count,stride);

Does someone had also such issues ? and found a solution to solve it ?

Thanks,

[Haonan Peng]

Hi Elodie Marret

Hi I also encountered such a problem. Do you ever manage to fix it?

Best regards,

Haonan

[Science Machine]

As I understand it, you need to build and compile the specific compression library against matlab's internal hdf5 library located in /path/to/matlab/R20xx/bin/glx??/libh5* . See also: https://www.mathworks.com/matlabcentral/answers/880033-build-hdf5-filter-plugins-on-linux-using-matlab-hdf5-shared-library-or-gnu-export-map

Answer 1

[Valerio4414]

I have had a similar issue in reading h5 Eiger detector data compressed with bitshuffle.

with MacOs 12.6 I have used mac port and installed the packages 'hdf5' and

'HDF5-External-Filter-Plugins'

which come with the following notes:

To enable use from Matlab, or other tools expecting the default HDF5 plugin path, create the required symbolic link:

mkdir -p /usr/local/hdf5/lib

ln -s /opt/local/lib/hdf5 /usr/local/hdf5/lib/plugin

This seems to be enough to be able to read the data.

Hope it helps.