real time video background replacement

Hello

I need to "cut out" a person from a real time webcam video and replace the background.

Based on the code of the following thread i tried to do the background replacement:

https://stackoverflow.com/questions/26958833/matlab-webcam-background-replacement

If i use the 2 webcams from my laptop, following error occurs:

winvideo: Image acquisition device is not accessible.
Insufficient system resources to perform the requested service.
Make sure no other objects or applications are accessing the same device.

If i try it with another usb camera (kinect), following error occurs:

Undefined operator '-' for input arguments of type 'videoinput'.
Error in Backgroundreplacement2 (line 32) diffMask = ((img1 - img2) > threshVal) | ((img2 - img1) > threshVal);

Here's my code:

   while 1

      % User Input:
      threshVal = 0;
      % Tolerance for difference between 'webcam' background and foreground 
      images

      %%%%%%%%%%%%
      % End User input

      % Read in some images 
      % img1 is the webcam background image

      cam = webcam('Intel(R) RealSense(TM) 3D Camera (Front F200) RGB');
      img1 = snapshot(cam);

      % bkgdImg is the image to replace the background of img1 with

      bkgdImg = imread('http://i.imgur.com/dIMct6u.jpg');

      % Resize bkgdImg so it is the same size as img1.  With two images from
      the same webcam this will not be necessary.

      bkgdImg = imresize(bkgdImg, [size(img1, 1), size(img1, 2)]); 

      % Draw on 'webcam' image to make it different
      % In actual algorithm, img2 would be a second image captured with the
      % webcam with the same background but different foregrounds, such as a
      % person now standing in front of camera.  

      img2 = videoinput('kinect',1);

      % Take difference of background and foreground images 
      % If difference is greater than threshVal, then add those pixels to the
      % mask.
      % The operation has to be done both ways because images are unsigned
      % integers and negative values are ignored.

      diffMask = ((img1 - img2) > threshVal) | ((img2 - img1) > threshVal);

      % Reduce mask by taking any value along 3rd dimension as logical true

      diffMask = any(diffMask, 3);

      % Replicate replacement background 

      bkgdImgReplace = bkgdImg;

      % In regions where mask is true (ie where background ~= foreground from
      % webcam) replace those pixels in the replacement background image with 
      the
      % foreground pixels

      bkgdImgReplace(repmat(diffMask, 1, 1, 3)) = img2(repmat(diffMask, 1, 1, 
      3));

      % Display

      figure()
      subplot(2, 2, 1)
      imshow(img1)
      title('Original Background')
      subplot(2,2,2)
      imshow(img2)
      title('Modified Background w/ New foreground')
      subplot(2,2,3)
      imshow(bkgdImg)
      title('Replacement Background')
      subplot(2,2,4)
      imshow(bkgdImgReplace);
      title('Replacement Background w/ New foreground')

      end

What do I have to change to make it work?

Thank's for any help!

Answer 1

[Arun Mathamkode]

Most hardware will not allow you to acquire data from multiple sources simultaneously. This is a limitation imposed by the hardware, and not by the Image Acquisition Toolbox. Maybe you can try to use 2 USB cams instead, it may help.

Refer to this answer provided by MathWorks Technical Support: Does the Image Acquisition Toolbox 1.0 (R13.0.1+) support simultaneous image capture from multiple cameras?

The second error occurs because the 'videoinput' creates a video input object, not an image. So in your case, img1 is an image but img2 is a video object. You may first need to extract frames from the video object using 'getsnapshot' and then find the difference with img1.

[david csernay]

Thanks for your reply. I managed to run the code with 'getsnapshot'. Unfortunately it runs very slow, so it's unusable.

I also tried it with 'peekdata' and 'getdata'. Same result.

Is there a way to do faster real time image processing?