how to read the data (.dat) inside online cluster (sftp) without downloading them.

Hi to Everyone and thank you in advance.

I would like to connect my local matlab (R2021b) to the online cluster (sftp protocol) in order to read and use data inside such cluster without downloading any data because of space on my hard disk.

By means of sftp() I'm able to reach the cluster, but so far the only way that I have found to read data is to download them.

Do you have any suggestions to fix this problem?

Nicola.

Answer 1

[Ayush Modi]

Hi Nicola,

I understand you want to work with data on a remote server through "sftp" without downloading them. "sftp" functionality in MATLAB only supports download, upload and delete operations on files.

However, you can try work around this limitation with a few different strategies:

1. Use SSH to run MATLAB commands on the remote cluster: If you have MATLAB installed on the remote cluster and it is possible to use SSH to execute commands, you could SSH into the cluster and run your MATLAB code there, working with the data directly on the cluster.
2. Mount the remote file system: If your operating system supports it, you could mount the remote file system as a network drive using SSHFS (SSH Filesystem). This would allow you to access the files as if they were on your local machine without actually downloading them. You would then use MATLAB's standard file I/O functions to read and write data.

I hope this resolves the issue you were facing.