Applying Threshold to Video Pixels

This code isn't working for me. I'm trying to limit video pixel values to 235, yet I am still getting pixels in the range 235 - 255 leaking through. Am I applying thresholding correctly?

I get a playable video with this code.

Vptr = VideoReader('Normal.mp4');
  img = zeros([Vptr.height,Vptr.width,3]);

    writer = VideoWriter('transcoded_xylophone.avi', 'Uncompressed AVI');
    writer.FrameRate = reader.FrameRate;
    open(writer);
    %Read and write each frame.
    while hasFrame(Vptr)
    img = readFrame(Vptr);

    % perform thresholding by logical indexing
    img(img>235) = 235;

    writeVideo(writer,img);

    end

    close(writer);

Answer 1

[Image Analyst]

It has nothing to do with double. The first call to set img with the zeros() function is totally ignored. The code works, though I did have to change reader.FrameRate to Vptr.FrameRate since the is no "reader" variable in your code. I ran it with the rhinos.avi demo video and it works fine, clipping the values to 235 as desired. See this code:

Vptr = VideoReader('Rhinos.avi');
writer = VideoWriter('delete_me.avi', 'Uncompressed AVI');
writer.FrameRate = Vptr.FrameRate;
open(writer);
% Read and write each frame.
frameCounter = 1;
while hasFrame(Vptr)
  thisFrame = readFrame(Vptr);  % This is a uint8 variable.
  % perform thresholding by logical indexing
  thisFrame(thisFrame>235) = 235;
  fprintf('For frame #%d, the max value = %d\n', frameCounter, max(thisFrame(:)))
  writeVideo(writer,thisFrame);
  frameCounter = frameCounter + 1;  
end
close(writer);

[Chris Clementson]

Question: If I use VideoReader to read a 4:2:0 YUV mp4 file, am I dealing with YUV values or does Matlab convert the samples to RGB?

[Image Analyst]

I haven't tried that. Can you attach a small such video? If you simply call

imshow(thisFrame);

does it look normal? If it does, it's an RGB image. If the colors look bizarre, then it's a YUV image where Y is displaying as red, U displaying as green, and V displaying as blue.

[Chris Clementson]

I'm using another program to plot luma levels. It calculates luma by:

Y = (R*Kr) + (G*Kg) + (B*Kb)

where Kr, Kg and Kb are the BT.709 luma coefficients.

I'm trying to clip off some camcorder sensor noise which reaches above 235. After applying the attached threshold code I'm still seeing excursions above 235. I can post some pictures later on.

See item 3.2 here: https://www.itu.int/dms_pubrec/itu-r/rec/bt/R-REC-BT.709-6-201506-I!!PDF-E.pdf

[Image Analyst]

That does not answer either of my questions.

And we'd really need your 'Normal.mp4' video, not some normal RGB pictures.

[Chris Clementson]

I'll answer those questions when I'm back on my regular computer :) (AFK now.)

[Chris Clementson]

Here are three relevant files:

http://www.chrisnology.info/videos/clip235out.avi

http://www.chrisnology.info/videos/Normal.MP4

http://www.chrisnology.info/videos/clip235out.JPG

What you're looking at in the jpg: The X axis corresponds to the X axis of the image. The Y axis represents the luma values of all the pixels in each column plotted one by one (not the sum). As you can see, there is still some excursion above digital 235 which I'm trying to get rid of.

I used your code, modifying only the input file name (see "DIG." scale on the right).

I added imshow to the code and we are dealing with RGB images. No R,G or B value may exceed 235. Here is how the math shakes out: Coeff.

R 0.2126 * 235 =49.961

G 0.7152 * 235 =168.072

B 0.0722 * 235 =16.967

Sum = luma = 235. This is what we want. We can make the luma 235 by making each of the RGB components 235.

[Chris Clementson]

Here is another file to look at. It was created by modifying the code thus:

thisFrame(thisFrame>=0) = 235; %set all pixels to 235

It sets all pixels greater than or equal to zero to 235.

http://www.chrisnology.info/videos/clip_all235_out.JPG

There is no digital noise, only a (hard-to-see) trace at digital 235. It proves the digital noise is not being introduced by the "scope" program.

[Image Analyst]

Not sure what luma is. Luminance? Lightness? Anyway, if you want to make sure that the weighted value of RGB doesn't exceed 235 then you need to convert to a color space where luminance is an axes. So you can use rgb2lab() and clip the L value to 235/255, or use rgb2hsv() and clip the v channel, or use rgb2ycbcr() and clip the y channel. Then use the companion function to convert the image back into RGB color space.

[Chris Clementson]

Luma = luminance, the "Y" in YUV.

As I posted previously, if none of the R, G or B components exceeds 235, the luminance will not exceed 235 after the coefficients are applied. It is a simple calculation.

I suspect these artifacts may be caused by the x264 encoder, not by matlab.

Answer 2

[Jose Marques]

Try to transform the image in a double matrix. readFrame gives a uint8 output.

[Jose Marques]

Bad answer! = )

Answer 3

[Chris Clementson]

Do you mean an array of doubles?

Video data is commonly uint8, i.e. 8-bit integers for RGB or YUV. There is no need to use doubles. All of the values are in the range 0 - 255.

[Jose Marques]

That's right... Look: just to be sure, try to add this code on yours:

 % perform thresholding by logical indexing
    img(img>235) = 235;
    max_value = max(max(max(img)))
    pause;