Writing Video Frames in Parallel

So I came up with a way to write video frames in parallel so that I could take advantage of all the cores in my computer only I keep getting a weird bug where I can only write videos up to 52.1 MB. Does anyone have any idea what might be going on? Below is my code I've been using to write video frames in parallel.

spmd(1,4)
open(videoWriter);
CountIndex = [0];
for i = labindex:numlabs:numFiles 
  if labindex == 1
  frame1 = imread(fullfile(workingDirectory, stackName, imageFiles(i).name));
  end
  if labindex == 2
  frame2 = imread(fullfile(workingDirectory, stackName, imageFiles(i).name));
  end
  if labindex == 3
  frame3 = imread(fullfile(workingDirectory, stackName, imageFiles(i).name));
  end
  if labindex == 4
  frame4 = imread(fullfile(workingDirectory, stackName, imageFiles(i).name));
  end

    while ~any(abs(CountIndex-(labindex-1))<1e-10)
    end
    if labindex == 1
    writeVideo(videoWriter,frame1);
    end
    if labindex == 2
    writeVideo(videoWriter,frame2);
    end
    if labindex == 3
    writeVideo(videoWriter,frame3);
    end
    if labindex == 4
    writeVideo(videoWriter,frame4);
    end
    CountIndex = [CountIndex labindex];

    if length(CountIndex) == length(numlabs)+1
        CountIndex = [0];
    end
end
end
end

[Walter Roberson]

That

    while ~any(abs(CountIndex-(labindex-1))<1e-10)
    end

is a nasty hack. Have you considered using one of the labs to write the data, with the others using labSend to send what they have read?

[Jacob Mevorach]

Hey Walter! I wasn't aware of a command named labSend and I've looked over the documentation but it's still not readily apparent to me how I might be able to use that to configure quick frame writing for a video. Might you be able to supply some help in that regard in the form of some descriptive pseudocode?

[Walter Roberson]

spmd
  if labindex == numlabs
    open(videoWriter);
    for i = 1 : numFiles
      frame = labReceive( 1 + mod(i - 1, numlabs-1) );
      writeVideo(videoWriter, frame);
    end
  else      
    for i = labindex:numlabs-1:numFiles
      frame = imread( imageFiles(i).name );
      labSend(frame, numlabs);
    end
  end
end

[Jacob Mevorach]

I would like to congratulate you on what I think is the most clever and functional snippet of code I've ever seen. Kudos to you sir, you have a real gift. This is beautiful. Thank you very much for you time.

Just as an edit: writing the code this way

spmd
done = 1;
if labindex == numlabs
  open(videoWriter);
  for i = 1 : numFiles
    frame = labReceive( 1 + mod(i - 1, numlabs-1) );
    writeVideo(videoWriter, frame);
    labsend(done, 1 + mod(i - 1, numlabs-1))
  end
else      
  for i = labindex:numlabs-1:numFiles
    frame = imread( imageFiles(i).name );
    labSend(frame, numlabs);
    labReceive(numlabs)
  end
end
end

will prevent the code from potentially writing a frame in the wrong order in the event that labSend returns before its corresponding labReceive.

[Walter Roberson]

You should not need that extra labSend / labReceive: the labReceive specifies which worker it is waiting to hear from and it cycles through the workers, so if (say) worker 3 has the data ready before worker #2, then the data from worker 3 will sit in the queue until the labReceive(3) is executed. If worker 3's data is not ready yet then labReceive(3) would wait for it.

The one thing I do not know, though, is what happens if you cue multiple items, whether a single labReceive will pick them all up.

Here is another way to avoid the labSend / labReceive that you added:

spmd
  if labindex == numlabs
    open(videoWriter);
    for i = 1 : numFiles
      frame = labReceive('any', i);
      writeVideo(videoWriter, frame);
    end
  else      
    for i = labindex:numlabs-1:numFiles
      frame = imread( imageFiles(i).name );
      labSend(frame, numlabs, i);
    end
  end
end

But there is also a completely different approach that does not use spmd: use parfeval()

p = parpool();
f(1:numFiles) = parallel.FevalFuture;
for i = 1  : numFiles
  f(i) = parfeval(p, @imread, 1, imageFiles(i).name );
end

open(videoWriter);
for i = 1 : numFiles
  frame = fetchOutputs(f(i));
  writeVideo(frame);
end

delete(f);

[Jacob Mevorach]

I put in the extra labreceive/labsend because labsend can potentially return before its corresponding labreceive completes its operation (see here: https://www.mathworks.com/help/distcomp/labsend.html). This has to do with the buffer size of the underlying MPI library's buffering behavior (see section 5.1.3 of this paper for a good explanation: http://paperity.org/p/17803450/matlab-r-a-language-for-parallel-computing). This would mean that if one was getting ready to send but two had already finished sometimes two would return early which would cause it to send frame n+3 rather than the frame n we wanted it to send which would in turn result in frame number n being dropped.

The SPMD example with the tags you sent seems like another solution to this problem.