visionhdl.DemosaicInterpolator

Construct full RGB pixel data from Bayer pattern pixel stream

Description

The visionhdl.DemosaicInterpolator System object™ constructs the full RGB pixel values from a Bayer pattern pixel stream. You can select a low-complexity bilinear interpolation or a moderate-complexity gradient-corrected bilinear interpolation. The object implements the calculations using hardware-efficient algorithms for HDL code generation.

• The object performs bilinear interpolation on a 3-by-3 pixel window using only additions and bit shifts.

• The object performs gradient correction on a 5-by-5 pixel window. The object implements the calculation using bit shifts, additions, and low-order canonical signed digit (CSD) multiplication.

To construct full RGB pixel data from a Bayer pattern pixel stream:

1. Create the visionhdl.DemosaicInterpolator object and set its properties.

2. Call the object with arguments, as if it were a function.

To learn more about how System objects work, see What Are System Objects?

Creation

Syntax

bayerInterpolator = visionhdl.DemosaicInterpolator

bayerInterpolator = visionhdl.DemosaicInterpolator(Name,Value)

Description

bayerInterpolator = visionhdl.DemosaicInterpolator creates a System object that interpolates RGB data from a Bayer pattern pixel stream.

bayerInterpolator = visionhdl.DemosaicInterpolator(Name,Value) sets properties using one or more name-value arguments. For example, 'SensorAlignment','RGGB' specifies the pattern of the RGB pixels in the input stream.

Properties

Unless otherwise indicated, properties are nontunable, which means you cannot change their values after calling the object. Objects lock when you call them, and the release function unlocks them.

If a property is tunable, you can change its value at any time.

For more information on changing property values, see System Design in MATLAB Using System Objects.

Algorithm — Algorithm to calculate the missing pixel values

'Gradient-corrected linear' (default) | 'Bilinear'

Algorithm to calculate the missing pixel values, specified as one of these values.

• 'Bilinear' — Average of the pixel values in the surrounding 3-by-3 neighborhood

• 'Gradient-corrected linear' (default) — Bilinear average, corrected for intensity gradient

SensorAlignment — Color sequence of the input pixels

'RGGB' (default) | 'GBRG' | 'GRBG' | 'BGGR'

Color sequence of the input pixels, specified as 'RGGB', 'GBRG', 'GRBG', or 'BGGR'. These values consist of four letters, R, G, or B, that correspond to the 2-by-2 block of pixels in the top-left corner of the input image. Specify the sequence in left-to-right, top-to-bottom order. For example, the default value, 'RGGB', represents an image with this pattern.

LineBufferSize — Size of line memory buffer

2048 (default) | positive integer

Size of the line memory buffer, specified as a positive integer. Choose a power of two that accommodates the number of active pixels in a horizontal line. If you specify a value that is not a power of two, the buffer uses the next largest power of two.

• When you set the InterpolationAlgorithm property to 'Bilinear', the object allocates 2-by-LineBufferSize memory locations.

• When you set the InterpolationAlgorithm property to 'Gradient-corrected linear', the object allocates 4-by-LineBufferSize memory locations.

Usage

Syntax

[pixelout,ctrlout] = bayerInterpolator(pixelin,ctrlin)

Description

[pixelout,ctrlout] = bayerInterpolator(pixelin,ctrlin) interpolates the missing color values of a Bayer pattern input pixel stream, and returns the next pixel value, pixelout, as a vector of RGB values. pixelin represents one pixel in a Bayer pattern image.

This object uses a streaming pixel interface with a structure for frame control signals. This interface enables the object to operate independently of image size and format and to connect with other Vision HDL Toolbox™ objects. The object accepts and returns a scalar pixel value and control signals as a structure containing five signals. The control signals indicate the validity of each pixel and its location in the frame. To convert a pixel matrix into a pixel stream and control signals, use the visionhdl.FrameToPixels object. For a description of the interface, see Streaming Pixel Interface.

Input Arguments

pixelin — Input pixel stream

scalar

Single input pixel, specified as an unsigned scalar integer. Images in the Bayer format have one color component for each pixel location. Select the sequence of R, G, and B pixels by using the SensorAlignment property.

You can simulate System objects with a multipixel streaming interface, but you cannot generate HDL code for System objects that use multipixel streams. To generate HDL code for multipixel algorithms, use the equivalent Simulink^® blocks.

The software supports double and single data types for simulation, but not for HDL code generation.

Data Types: uint | fi(0,W,F) | single | double

ctrlin — Control signals accompanying input pixel stream

pixelcontrol structure

Control signals accompanying the input pixel stream, specified as a pixelcontrol structure containing five logical data type signals. The signals describe the validity of the pixel and its location in the frame. For more details, see Pixel Control Structure.

Data Types: struct

Output Arguments

pixelout — Output pixel stream

scalar

Single output pixel, returned as a three-element vector of values in RGB color space.

The data type of pixelout is the same as the data type of pixelin.

Data Types: uint | fi(0,W,F) | single | double

ctrlout — Control signals accompanying output pixel stream

pixelcontrol structure

Control signals accompanying the output pixel stream, returned as a pixelcontrol structure containing five logical data type signals. The signals describe the validity of the pixel and its location in the frame. For more details, see Pixel Control Structure.

Data Types: struct

Object Functions

To use an object function, specify the System object as the first input argument. For example, to release system resources of a System object named obj, use this syntax:

release(obj)

Common to All System Objects

step	Run System object algorithm
release	Release resources and allow changes to System object property values and input characteristics
reset	Reset internal states of System object

Examples

Demosaic Bayer Pattern Image

This example shows how to construct full RGB pixel data from a Bayer pattern thumbnail image.

Set the dimensions of the test image. Load the source image file. This image is represented in a Bayer pattern, meaning each pixel is represented by one value, alternating green values with red and blue values. Select a portion of the image matching the desired test size. These offsets select the face of the person in the image.

frmActivePixels = 256;
frmActiveLines = 192;
frmOrig = imread('mandi.tif');
frmInput = frmOrig(900:899+frmActiveLines,2350:2349+frmActivePixels);
figure
imshow(frmInput)
title 'Input Image'

Create a serializer object and specify the size of the inactive pixel regions.

frm2pix = visionhdl.FrameToPixels(...
      'NumComponents',1, ...
      'VideoFormat','custom', ...
      'ActivePixelsPerLine',frmActivePixels, ...
      'ActiveVideoLines',frmActiveLines, ...
      'TotalPixelsPerLine',frmActivePixels+10, ...
      'TotalVideoLines',frmActiveLines+10, ...
      'StartingActiveLine',6, ...     
      'FrontPorch',5);

Create an interpolator object. Specify the sequence of color values matching the 2-by-2 pixels in the top-left corner of the image.

BayerInterpolator = visionhdl.DemosaicInterpolator(...
   'SensorAlignment','RGGB');

Serialize the test image. pixIn is a vector of pixel values. ctrlIn is a vector of control signal structures.

[pixIn,ctrlIn] = frm2pix(frmInput);

Set up variables, and then generate the RGB triplet for each pixel in the stream. This example prints a progress message every 32 lines.

[pixels,lines,numPixelsPerFrame] = getparamfromfrm2pix(frm2pix);
ctrlOut = repmat(pixelcontrolstruct,numPixelsPerFrame,1);
pixOut = zeros(numPixelsPerFrame,3,'uint8');
lineCount = 1;
for p = 1:numPixelsPerFrame 
    if ctrlIn(p).hEnd
         lineCount = lineCount+1;
         if mod(lineCount,32)==0
              fprintf('Processing line #%d\n',lineCount)
         end
    end
    [pixOut(p,:),ctrlOut(p)] = BayerInterpolator(pixIn(p),ctrlIn(p));
end

Processing line #32
Processing line #64
Processing line #96
Processing line #128
Processing line #160
Processing line #192

Create a deserializer object with a format that matches that of the serializer. Convert the pixel stream to an image frame, and then display the result.

pix2frm = visionhdl.PixelsToFrame(...
      'NumComponents',3, ...
      'VideoFormat','custom', ...
      'ActivePixelsPerLine',frmActivePixels, ...
      'ActiveVideoLines',frmActiveLines, ...
      'TotalPixelsPerLine',frmActivePixels+10);
[frmOutput,frmValid] = pix2frm(pixOut,ctrlOut);
if frmValid
    figure
    imshow(frmOutput)
    title 'Output Image'
end

Algorithms

This object implements the algorithms described on the Demosaic Interpolator block reference page.

Extended Capabilities

C/C++ Code Generation

Generate C and C++ code using MATLAB® Coder™.

This System object supports C/C++ code generation for accelerating MATLAB^® simulations, and for DPI component generation. For more information about acceleration, see Accelerate Pixel-Streaming Designs Using MATLAB Coder. For more information about DPI component generation, see Considerations for DPI Component Generation with MATLAB (HDL Verifier).

HDL Code Generation

Generate VHDL, Verilog and SystemVerilog code for FPGA and ASIC designs using HDL Coder™.

To generate HDL code from Vision HDL Toolbox System objects, see Design Hardware-Targeted Image Filters in MATLAB.

Version History

Introduced in R2015a

R2022a: Two pixels-per-clock streaming

The object now supports multipixel streams that have 2 pixels per clock cycle.

R2021b: Multipixel streaming

The object now supports multipixel streams. The object accepts an input vector of NumPixels-by-1 values and returns an output matrix of NumPixels-by-3 values, where NumPixels is 4 or 8. The ctrl argument remains scalar, and the control signals in the pixelcontrol structure apply to all pixels in the vector.

R2018b: Improved line buffer

The internal line buffer in this object now handles bursty data, that is, noncontiguous valid signals within a pixel line. This implementation uses fewer hardware resources due to improved padding logic and native support for kernel sizes with an even number of lines. This change affects the visionhdl.LineBuffer object and objects that use an internal line buffer.

The latency of the line buffer is now reduced by a few cycles for some configurations. You might need to rebalance parallel path delays in your designs. A best practice is to synchronize parallel paths in your designs by using the pixel stream control signals rather than by inserting a specific number of delays.