Copying files from Matlab without a system call

Hi everybody,

I'm having a problem running a compiled matlab script. The script includes some "copyfile" commands, which normally shouldn't be a problem. The trouble is, that the compiled script is distributed among slave nodes in a cluster.

As it turns out, system calls inside a script can be a hell of a problem, as it seems to be the case with the parallel cluster I am using. ( I got the lethal error Warning: MATLAB was unable to open a pseudo-tty: No such file or directory [2,1] ) which basically means of my syscalls will crash the code.

Can anybody think of some way of coping/moving files that does not require an implicit syscall? thanks, Daniel

[Fangjun Jiang]

Did you use MATLAB built-in function copyfile() or use system('copy ...')?

[Daniel]

Hi Fangjun,

thanks for te comment. Initially I used unix('cp .. '); didn't work... Then I did try with the built-in function copyfile, didn't work either. I suspect copyfile is just a wrapper and internally uses syscalls.

Answer 1

[Walter Roberson]

I suggest that you investigate what your MATLAB_SHELL and SHELL environment variables are, in case they are set to something unexpected (see the documentation for the unix() command for more information.)

My suspicion is that on the slaves, something in /etc/profile or in your ~/.profile or ~/.bashrc or the like is unconditionally attempting to do an operation on a tty, such as by trying to do

stty erase "^H"

If you have a command like that in your shell startup script, you need to make it conditional upon a ptty existing, such as by testing with

if -t 0
  stty erase "^H"
fi

(Note those are at the shell level, not the MATLAB level.)

[Daniel]

interesting. I've checked that the compiled script performs rightly in the master node. I'll check if the startup shells of master and slaves differ in some tty setting...

thanks!

[Walter Roberson]

Sometimes it gets down to the master node having a video card (even if just for diagnostic purposes) and the slaves not having video cards; in such cases, an attempted pty operation on the master might refer to the "graphics console" and succeed because the console exists, but the same operation fails on a slave that doesn't have a graphics card.

Shell script initialization can be a bit of a "black art" in distributed environments. (Sorry, I never lend out my athame!)

[Daniel]

that makes sense! Checking it with my system administrator,who is at a loss in this case... thanks!

Answer 2

[Jason Ross]

Can you use cluster commands (outside of MATLAB) to copy the files as part of your submission?

[Daniel]

thanks for the suggestion. In this case I'd rather not, as it would involve a major rewriting on the code :-(

[Jason Ross]

Well, it was worth a try.

You might want to try using pmode to see if that will help you diagnose your issue. It's a interactive shell that will connect to each worker lab on the cluster from the MATLAB command window. I suspect the overall resolution to do with the worker OS configuration, as Walter has already mentioned, but this might be a better environment for debugging.

[Jason Ross]

You might also want to take a look at FileDependencies and PathDependencies, as well, to see if they work for your cluster and what you are trying to do.

[Jason Ross]

Doc refs:

Pmode: http://www.mathworks.com/help/toolbox/distcomp/bqrtu80.html

FileDependencies: http://www.mathworks.com/help/toolbox/distcomp/filedependencies.html

PathDependencies: http://www.mathworks.com/help/toolbox/distcomp/pathdependencies.html

[Daniel]

thanks Jason, that's a new perspective. Actually I'm not using any of the functionalities of Parallel toolbox of Matlab (I just distribute compiled executables in the cluster). I'll check it.