Parallel Computing Question

Hi, I am a noob in matlab anf I am trying to parallelize loading of an array file.

So suppose the array size is X by Y. I would like to load this file to "n" matlab workers giving X/n by Y to 1st worker and X/n by Y to another and so on.... Does anyone know how can I achieve that since the file is too large and load of that file in a distributed way can save me loads of time.

I tried pmode but since it loads the entire data on each worker and as my data file is huge, it takes a lot of time. So i want to load only X/n by Y on each worker where n is the total number of workers.

So I am trying something like this for my application:

1. Start n worker nodes 2. load X/n by Y data on each worker node 3. Output = function (X/n, Y) 4. Final Output = Gather (Output) 5. Stop all worker nodes.

Do let me know if you can help me in any possible way.

Thanks

Answer 1

[the cyclist]

I don't have experience with the parallel computing toolbox, but I speculate that the relatively new matfile() command, which allows loading of partial arrays, maybe be helpful to you.

Answer 2

[Jill Reese]

You might take a look at the DLOAD functionality that is part of the Parallel Computing Toolbox. This function loads distributed arrays and composites from disk, but if you also supply the -scatter option, then DLOAD will also distribute non-distributed data where possible.