No parallel read/write with NetCDF4?

Parallel reading and writing of NetCDF version 4 files does not seem to work in Matlab. Is this correct?

Previously, I would create a file with,

netcdf.create('myfile.nc', bitor(netcdf.getConstant('SHARE'),netcdf.getConstant('WRITE')));

which creates a NetCDF version 3, and allows me to read that file while it is being written (think multi-hour model run). However, I need some new features only available in NetCDF version 4, so I switched to,

netcdf.create('myfile.nc', netcdf.getConstant('NETCDF4'));

which gives me the new features I want, but now I can no longer read the NetCDF file as it is being written! To test this you can call,

netcdf.open('myfile.nc', bitor(netcdf.getConstant('SHARE'),netcdf.getConstant('WRITE')));

from a new Matlab instance and the former will work, but the later will fail.

Looking at the NetCDF documentation, it looks like this should be possible, provided HDF5 is compiled with the parallel flag. Does Matlab simply not have this enabled? Or am I missing an option?

Thank you.

edit: Added code to reproduce.

[Walter Roberson]

Would you perhaps need to use

netcdf.create('myfile.nc', bitor(netcdf.getConstant('NETCDF4'), bitor(netcdf.getConstant('SHARE'),netcdf.getConstant('WRITE'))));

[Jeffrey]

@Walter Roberson Thanks for checking this out.

Unfortunately no, the SHARE option is inert with NetCDF4 files, so it doesn't make a difference. (And I think the WRITE option is implict when creating files, so I should have left it off in the first command).

[Walter Roberson]

It looks like NETCDF4 switched to a new open routine instead of a bit flag -- nc_open_par() .

But it doesn't look like MATLAB supports that.

[Jeffrey]

Thanks @Walter Roberson, yes, nc_open_par() does look like the intended solution. The function is in the included netcdf library with Matlab,

$ nm -gU /Applications/MATLAB_R2023a.app/bin/maci64/libnetcdf.19.dylib | grep nc_open_par
0000000000003340 T _nc_open_par
0000000000003350 T _nc_open_par_fortran

but looking at the api, it would require setting up MPI. So this is probably something Mathworks will have to implement.

It's kind of an odd regression... I'll have to see what other ideas I can come up with.

[Walter Roberson]

MATLAB Parallel Computing Toolbox is built with MPI, which is used for spmd .

https://www.mathworks.com/help/parallel-computing/mpisettings.html

https://www.mathworks.com/help/matlab-parallel-server/use-different-mpi-builds-on-unix-systems.html