is it faster to load a big file (198MB) once and pass it onto a function in a parfor loop or let the function in parfor loop to load it every iteration?

조회 수: 1(최근 30일)
Dear All,
is it faster to load a big .mat file (198MB) once and pass it onto a function in a parfor loop or let the function in parfor loop to load it every iteration? My hunch is that to load it once (which take about 10min) and let it pass to the function. But I am not sure. Also, are there faster ways to load .mat file? Currently, I am simply using the "load" function.
Another question is that will the script run faster if I put all the codes in a single m file as compared to separating them into different functions (also m files). The readability will improve but I am not sure if the computation efficiency is significantly impacted. Thanks! best, Gene
  댓글 수: 1
Jason Ross
Jason Ross 2013년 2월 7일
편집: Jason Ross 2013년 2월 7일
Are you running everything locally on one machine, or are your workers distributed among several machines? Do you plan on running this on a cluster anytime soon? Is there a shared file system if you are planning on doing so?

댓글을 달려면 로그인하십시오.

답변(2개)

Jason Ross
Jason Ross 2013년 2월 7일
You might find Edric's FEX submission worth reviewing:

Dan K
Dan K 2013년 2월 7일
A few things...
1 it should not take minutes to load a file that's only 200MB, so you may want to look at how you're loading it.
2. Yes, it is faster to load once then pass it around, since RAM is faster than a HD.
3. It doesn't matter if it's one file or many, but.... You'll get much more benefit by converting your script into a function

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by