Matlab parfor saves and loads temporary variables during execution!
조회 수: 3(최근 30일)
The parallel pool implementation appears to save and load variables using effectively the normal "save" and "load" channels when making copies to be passed to the workers. This is really bad because if someone has a saveobj() implemented in a class object the state of the variable could be modified during execution and depending on the circumstances could lead to "unexplained" crashes.
Lets say I have a class object that needs a large amount of temporary data during execution. When saved I would naturally want to get rid of this temporary data. A convenient way to do this is to write a saveobj that clears the temporary data within the class. If, however, the object is saved and reloaded during execution as with parfor, there are problems.
Why is the copy not made internally?
Matt J 2018년 7월 21일
편집: Matt J 2018년 7월 21일
I don't know why authoritatively, but it seems to me that this is only a danger if you are unaware of this behavior on the part of parfor. If you are aware of it, you would surely write a loadobj() method to restore the temporary data when reloaded. And, this would be better than broadcasting copies of the temporary data to all the workers. The latter is done serially and would take a lot more time.
Incidentally, have you verified that saveobj() is triggered in this scenario just as in an ordinary call to save()?
Edric Ellis 2018년 7월 23일
The workers executing the body of any parfor loop are separate MATLAB processes, so the only reliable way that variables existing on the client can be sent to those workers is by doing something equivalent to calling save on the client, and then load on the workers. (The same procedure is used, but no files are created on disk).
I must admit, it's not clear to me what the benefit is of writing a saveobj that cannot be reversed by a loadobj. (Also, have you considered using Transient fields in your class?)