visionhdl.FrameToPixels

Convert frame-based video to pixel stream

Description

The visionhdl.FrameToPixels System object™ converts color or grayscale frame-based video to a pixel stream and control structure. The control structure indicates the validity of each pixel and its location in the frame. The pixel stream format can include padding pixels around the active frame. You can configure the frame and padding dimensions by selecting a common video format or by specifying custom dimensions. For details about the pixel stream format, see Streaming Pixel Interface.

Use this object to generate input for a function targeted for HDL code generation. This object itself does not support HDL code generation.

If your design converts frames to a pixel stream and later converts the stream back to frames, specify the same video format for the FrameToPixels object and the PixelsToFrame object.

To convert frame-based video to a pixel stream:

1. Create the visionhdl.FrameToPixels object and set its properties.

2. Call the object with arguments, as if it were a function.

To learn more about how System objects work, see What Are System Objects?

Creation

Syntax

F2P = visionhdl.FrameToPixels(Name,Value)

Description

F2P = visionhdl.FrameToPixels(Name,Value) returns a System object that serializes input frames into a pixel stream. Set properties using one or more name-value pairs. Enclose each property name in single quotes.

Properties

Unless otherwise indicated, properties are nontunable, which means you cannot change their values after calling the object. Objects lock when you call them, and the release function unlocks them.

If a property is tunable, you can change its value at any time.

For more information on changing property values, see System Design in MATLAB Using System Objects.

NumComponents — Number of values used to represent each pixel

1 (default) | 3 | 4

Number of values used to represent each pixel, specified as 1, 3, or 4 components.

• For grayscale video, set this property to 1.

• For color video, for example, {R,G,B} or {Y,Cb,Cr}, set this property to 3.

• For color video with an alpha channel for transparency, set this property to 4.

The visionhdl.FrameToPixels object returns a P-by-NumComponents matrix, where P is the total number of pixels in the padded frame.

Dependencies

When NumComponents is greater than 1, you must set the NumPixels property to 1.

NumPixels — Number of pixels streamed per time step

1 (default) | 2 | 4 | 8

Number of pixels transferred on the streaming interface for each cycle, specified as 1, 2, 4, or 8. To enable multipixel streaming and increase throughput for high-resolution or high-frame-rate video, set this property to 2, 4, or 8. The visionhdl.FrameToPixels object returns a P-by-NumPixels matrix, where P is the total number of pixels in the padded frame.

Note

You can simulate System objects with a multipixel streaming interface, but you cannot generate HDL code for System objects that use multipixel streams. To generate HDL code for multipixel algorithms, use the equivalent Simulink^® blocks.

Dependencies

When NumPixels is greater than 1, you must set the NumComponents property to 1.

VideoFormat — Dimensions of active and inactive regions in video frame

'1080p' (default) | '240p' | '480p' | '480pH' | '576p' | '720p' | '768p' | '1024p' | '1200p' | '2KCinema' | '4KUHDTV' | '8KUHDTV' | 'Custom'

Dimensions of active and inactive regions of a video frame. To select a predefined format, specify the VideoFormat property as one of the options in the first column of the table. For a custom format, set VideoFormat to 'Custom', and specify the dimension properties as integers. The frame dimensions are indicated in the diagram.

Video Format	Active Pixels Per Line	Active Video Lines	Total Pixels Per Line	Total Video Lines	Starting Active Line	Ending Active Line	Front Porch	Back Porch
240p	320	240	402	324	1	240	44	38
480p	640	480	800	525	36	515	16	144
480pH	720	480	858	525	33	512	16	122
576p	720	576	864	625	47	622	12	132
720p	1280	720	1650	750	25	744	110	260
768p	1024	768	1344	806	10	777	24	296
1024p	1280	1024	1688	1066	42	1065	48	360
1080p (default)	1920	1080	2200	1125	42	1121	88	192
1200p	1600	1200	2160	1250	50	1249	64	496
2KCinema	2048	1080	2750	1125	42	1121	639	63
4KUHDTV	3840	2160	4400	2250	42	2201	88	472
8KUHDTV	7680	4320	8800	4500	42	4361	88	1032
Custom	User- defined	User- defined	User- defined	User- defined	User- defined	User- defined	User- defined	User- defined

Note

When using a custom format, the properties you enter for the active and inactive dimensions of the image must add up to the total frame dimensions.

For the horizontal direction, TotalPixelsPerLine must be greater than or equal to FrontPorch + ActivePixelsPerLine. The object calculates BackPorch = TotalPixelsPerLine − FrontPorch − ActivePixelsPerLine.

For the vertical direction, TotalVideoLines must be greater than or equal to StartingActiveLine + ActiveVideoLines − 1. The object calculates EndingActiveLine = StartingActiveLine + ActiveVideoLines − 1.

If you specify a format that does not conform to these rules, the object reports an error.

Note

When using a custom format, ActivePixelsPerLine must be greater than 1. Also, set the horizontal blanking interval, or BackPorch + FrontPorch, according to these guidelines.

• The total of BackPorch + FrontPorch must be at least 2 times the largest kernel size of the algorithm in the objects following the visionhdl.FrameToPixels object. If the kernel size is greater than 4, the total porch must be at least 8 pixels.

• The BackPorch property value must be at least 6 pixels. This property is the number of inactive pixels before the first valid pixel in a frame.

For more information on blanking intervals, see Configure Blanking Intervals.

Note

When using multipixel streaming (NumPixels > 1), these requirements apply.

• The video format must have horizontal dimensions divisible by the NumPixels property value. The horizontal dimensions are set by these properties: ActivePixelsPerLine, TotalPixelsPerLine, FrontPorch, and BackPorch. Standard video protocols 480p, 768p, 1024p, 1080p, 1200p, 4k UHD, and 8k UHD support NumPixels equal to 4 or 8.

• The minimum input frame size for multipixel streaming is 18 rows by 32 columns.

• Choose your kernel size and ActivePixelsPerLine such that ActivePixelsPerLine/NumPixels is at least the kernel width.

Usage

Syntax

[pixels,ctrlOut] = F2P(frm)

Description

[pixels,ctrlOut] = F2P(frm) converts the input image matrix, frm, to a vector of pixel values, pixels, and an associated vector of control structures, ctrlOut. The control structure indicates the validity of each pixel and its location in the frame. The output pixels include padding around the active image, specified by the VideoFormat property.

For details about the pixel stream format, see Streaming Pixel Interface.

Input Arguments

frm — Input image

matrix

Input image, specified as an ActiveVideoLines-by-ActivePixelsPerLine-by-NumComponents matrix, where:

• ActiveVideoLines is the height of the active image.

• ActivePixelsPerLine is the width of the active image.

• NumComponents is the number of components used to express a single pixel.

Set the size of the active image using the VideoFormat property. If the dimensions of frm do not match the dimensions specified by VideoFormat, the object returns a warning.

Data Types: uint | int | logical | fi | double | single

Output Arguments

pixels — Pixel values

matrix

Pixel values, returned as a P-by-NumComponents matrix or P-by-NumPixels matrix, where:

• P is the total number of pixels in the padded image, which is TotalPixelsPerLine × TotalVideoLines.

• NumComponents is the number of components used to express a single pixel.

• NumPixels is the number of pixels transferred on the streaming interface per cycle. When NumPixels is greater than 1, you must set NumComponents to 1.

  Note

  You can simulate System objects with a multipixel streaming interface, but you cannot generate HDL code for System objects that use multipixel streams. To generate HDL code for multipixel algorithms, use the equivalent Simulink blocks.

Set the size of the padded image using the VideoFormat property. The data type of the pixel values is the same as im.

ctrlOut — Pixel stream control signals

vector of structures

Control structures associated with the output pixels, returned as a P-by-1 vector. P is the total number of pixels in the padded image, which is TotalPixelsPerLine × TotalVideoLines. Each structure contains five control signals indicating the validity of the pixel and its location in the frame. For multipixel streaming, the control signals apply to each set of NumPixels values. See Pixel Control Structure.

Object Functions

To use an object function, specify the System object as the first input argument. For example, to release system resources of a System object named obj, use this syntax:

release(obj)

Common to All System Objects

step	Run System object algorithm
release	Release resources and allow changes to System object property values and input characteristics
reset	Reset internal states of System object

Examples

Convert Between Full-Frame and Pixel-Streaming Data

This example converts a custom-size grayscale image to a pixel stream. It uses the visionhdl.LookupTable object to obtain the negative image. Then it converts the pixel stream back to a full-frame image.

Load the source image from a file. Select a portion of the image matching the desired test size.

frmOrig = imread('rice.png');
frmActivePixels = 64;
frmActiveLines = 48;
frmInput = frmOrig(1:frmActiveLines,1:frmActivePixels);
figure
imshow(frmInput,'InitialMagnification',300)
title 'Input Image'

Create a serializer object and specify the size of the inactive pixel regions.

frm2pix = visionhdl.FrameToPixels( ...
      'NumComponents',1, ...
      'VideoFormat','custom', ...
      'ActivePixelsPerLine',frmActivePixels, ...
      'ActiveVideoLines',frmActiveLines, ...
      'TotalPixelsPerLine',frmActivePixels+10, ...
      'TotalVideoLines',frmActiveLines+10, ...
      'StartingActiveLine',6, ...     
      'FrontPorch',5);

Create a lookup table (LUT) object to generate the negative of the input image.

tabledata = linspace(255,0,256);
inverter = visionhdl.LookupTable(tabledata);

Serialize the test image by calling the serializer object. pixIn is a vector of intensity values. ctrlIn is a vector of control signal structures.

[pixIn,ctrlIn] = frm2pix(frmInput);

Prepare to process pixels by preallocating output vectors.

[~,~,numPixelsPerFrame] = getparamfromfrm2pix(frm2pix);
pixOut = zeros(numPixelsPerFrame,1,'uint8');
ctrlOut = repmat(pixelcontrolstruct,numPixelsPerFrame,1);

For each pixel in the stream, look up the negative of the pixel value.

for p = 1:numPixelsPerFrame  
    [pixOut(p),ctrlOut(p)] = inverter(pixIn(p),ctrlIn(p));
end

Create a deserializer object with a format that matches the format of the serializer. Convert the pixel stream to an image frame by calling the deserializer object. Display the resulting image.

pix2frm = visionhdl.PixelsToFrame( ...
      'NumComponents',1, ...
      'VideoFormat','custom', ...
      'ActivePixelsPerLine',frmActivePixels, ...
      'ActiveVideoLines',frmActiveLines, ...
      'TotalPixelsPerLine',frmActivePixels+10);
[frmOutput,frmValid] = pix2frm(pixOut,ctrlOut);
if frmValid
    figure
    imshow(frmOutput,'InitialMagnification',300)
    title 'Output Image'
end

Version History

Introduced in R2015a

R2022a: Change in default frame dimensions

The default values of the custom format have changed. In previous releases, the default dimensions matched the 1080p format. Starting in this release, the default values match the default values of the Frame To Pixels and Pixels To Frame blocks, which define a small custom frame as shown in this code.

a = visionhdl.FrameToPixels('VideoFormat','Custom')

a = 

  visionhdl.FrameToPixels with properties:

          NumComponents: 1
              NumPixels: 1
            VideoFormat: 'Custom'
    ActivePixelsPerLine: 32
       ActiveVideoLines: 18
     TotalPixelsPerLine: 42
        TotalVideoLines: 28
     StartingActiveLine: 6
             FrontPorch: 5

This change means you might need to specify additional dimensions for custom formats where previously the default value resulted in a valid format.

The object now updates frame dimension properties when you select a standard format. This change means you can refer to the object properties to determine the dimensions being used by the format.

a = visionhdl.FrameToPixels('VideoFormat','480p')

a = 

  visionhdl.FrameToPixels with properties:

    NumComponents: 1
        NumPixels: 1
      VideoFormat: '480p'

TPPL480 = a.TotalPixelsPerLine

ans =

   800

R2022a: Two pixels-per-clock streaming

The object can now generate multipixel streams that have 2 pixels per clock cycle.

You can simulate System objects with a multipixel streaming interface, but you cannot generate HDL code for System objects that use multipixel streams. To generate HDL code for multipixel algorithms, use the equivalent Simulink blocks.

R2020a: Multipixel-multicomponent streaming

The object can now generate multipixel-multicomponent streams.

You can simulate System objects with a multipixel streaming interface, but you cannot generate HDL code for System objects that use multipixel streams. To generate HDL code for multipixel algorithms, use the equivalent Simulink blocks.

R2019b: Multipixel streaming

The object can now generate multipixel streams. These streams have 4 or 8 pixels per clock cycle. You can set the NumPixels property to 4 or 8. With this setting, the output of the object is a vector of 4 or 8 pixel values on each time step.

Video formats for multipixel streams must have horizontal dimensions divisible by the NumPixels property value. These horizontal dimensions are set by the following properties: ActivePixelsPerLine, TotalPixelsPerLine, FrontPorch, and BackPorch. Standard video protocols 480p, 720p, 1080p, and 4k UHD support either 4 or 8 pixels at a time.

You can simulate System objects with a multipixel streaming interface, but you cannot generate HDL code for System objects that use multipixel streams. To generate HDL code for multipixel algorithms, use the equivalent Simulink blocks.