How to speed up frame-by-frame image processing for writing a video?

Hello,

I have a large video (avi) that I want to read, process each frame, crop, and save a processed video. Here is my example script, which takes extraordinally long time to run (~35 seconds/100 frames). Is there more efficient way to read, process, and write frames to a video?

% Read video
fname = 'vid.avi';
reader=VideoReader(fname);
n_frames =  reader.NumFrames;
crop_position = [1000, 300, 149, 99];

% Write video
writer = VideoWriter('cropped_vid', 'MPEG-4');
writer.Quality = 95;
writer.FrameRate = reader.FrameRate;

% process frames
for i=1:n_frames
    img = read(reader,i);
    img = rgb2lab(img);
    img = imcomplement(imreducehaze(imcomplement(img(:,:,1) ./ 100),'ContrastEnhancement','boost'));
    img = medfilt2(imcrop(img,crop_mask),[5,5]);
    img = imcomplement(imreducehaze(imcomplement(img)));
    writeVideo(writer,img);
end
close(writer);
toc

Thanks!

[Walter Roberson]

It would be interesting to compare timing to https://www.mathworks.com/help/vision/ref/vision.videofilereader-system-object.html

[NeuronDB]

vision.videoreader does speed up reading the frames, but the most of the time is in the code is spent in rgb2lab conversion :( with vision block, it the time comes down to ~ 32 seconds/100 frames

[Walter Roberson]

You might be able to reduce overhead by digging into the conversion functions. For the default white point,

format long g
M = images.color.internal.linearRGBToXYZTransform(true).'

M = 3×3

0.412456439089692 0.212672851405623 0.0193338955823293 0.357576077643909 0.715152155287818 0.119192025881303 0.180437483266399 0.0721749933065596 0.950304078536368

appears to be a matrix to convert RGB to XYZ -- but there is some stuff in there about "encoded" or "unecoded" values that I do not understand at the moment.

The logic for XYZ to LAB takes more lines, and again I am missing elements about "encoded" or unencoded values (I am not familiar with the terminology)

The idea here is that the number of levels of calling and creation of anonymous functions in using rgb2lab might perhaps be adding significant overhead, so maybe you could improve performance by doing at least part of it "by hand".

[Yair Altman]

The first thing that I suggest to do is to run your script in the Matlab Profiler, for example by clicking the <Run and Time> button in the Editor. This will tell you where most of the run-time is spent:

1. perhaps it's reading the input video with read(reader,1)?
2. perhaps it's in the rgb2lab conversion?
3. perhaps it's the processing of each frame?
4. perhaps it's writing to the output video using writeVideo?

Based on what you discover, it will be easier to focus your speed-up efforts. For example, perhaps you can vectorize some operations rather than processing the frames one-by-one; or perhaps you can move the the read/write parts outside the loop for faster I/O; or perhaps you can improve the processing by using a custom conversion. Before you know where the main bottleneck is, the efforts will be wasted.

[Walter Roberson]

User already said, "the most of the time is in the code is spent in rgb2lab conversion"

Answer 1

[yanqi liu]

yes，sir，may be split image process and video write，such as process image to files，and then read files and write to video，so it will avoid video write flush step

[NeuronDB]

unfortunately this doesnt change the processing time