How do I access netCDF files remotely?

Hi,

I would like to read data in from very large netCDF file stored on a remote server. Currently I am copying the file to my local computer (using rsync) and then opening it (using netcdf.open, etc.). However, this creates multiple redundant copies of the same large file (one on the remote machine, one on my machine, and another on my cloud-based back up). I would therefore like any advice on how to directly read the remote file into the MATLAB workspace (without copying it to the local machine).

Is this possible?

Help appreciated,

Jeremy

Answer 1

[Jan]

Store the file to a shared disk, e.g. mount the remote disk locally. Then the usual file access works fine - but slowly. If the file is really huge, e.g. some GB, accessing it through the network will consume much more time than accessing a local copy. The speed difference depends on the reading function and the network speed, but even a factor of 100 is possible.

I'd create a file copy in the local TEMP folder, which is hopefully not duplicated in the cloud-based backup.

[Jeremy]

Hi Jan,

Yes this works. I mounted my local machine to the remote machine using sshfs (via MacFuse and MacFusion since the local machine is a Mac...). My network is pretty good, so speed not a major issue.

Thanks a lot for your help,

Jeremy