Design Hardware-Targeted Image Filters in MATLAB
This example shows how to design a hardware-targeted image filter using Vision HDL Toolbox™ objects.
The key features of a model for hardware-targeted video processing in MATLAB® are:
Streaming pixel interface: System objects in Vision HDL Toolbox use a streaming pixel interface. Serial processing is efficient for hardware designs, because less memory is required to store pixel data. The serial interface enables the object to operate independently of image size and format and makes the design more resilient to video timing errors. For further information, see Streaming Pixel Interface.
Function targeted for HDL code generation: Once the data is converted to a pixel stream, you can design a hardware model by selecting System objects from the Vision HDL Toolbox libraries. The part of the design targeted for HDL code generation must be in a separate function.
Conversion to frame-based video: For verification, you can display frame-based video, or you can compare the result of your hardware-compatible design with the output of a MATLAB frame-based behavioral model. Vision HDL Toolbox provides a System object™ that enables you to deserialize the output of your design.
Read an image file into the workspace. This sample image contains 256×256 pixels. Each pixel is a single
uint8 value representing intensity. To reduce simulation speed while testing, select a thumbnail portion of the image.
Simulating serial video in the MATLAB interpreted language can be time-consuming. Once you have debugged the design with a small image size, use MEX code generation to accelerate testing with larger images. See Accelerate Pixel-Streaming Designs Using MATLAB Coder.
origIm = imread('rice.png'); origImSize = size(origIm) imActivePixels = 64; imActiveLines = 48; inputIm = origIm(1:imActiveLines,1:imActivePixels); figure imshow(inputIm,'InitialMagnification',300) title 'Input Image'
origImSize = 256 256
visionhdl.FrameToPixels System object converts framed video to a pixel stream and control structure. This object provides input for a function targeted for HDL code generation, but it does not itself support HDL code generation.
To simulate with a standard video format, choose a predefined video padding format to match your input source. To simulate with a custom-sized image, choose dimensions of inactive regions to surround the image. This tutorial uses a custom image. The properties of the
visionhdl.FrameToPixels object correspond to the dimensions in the diagram.
visionhdl.FrameToPixels object and set the image properties. The image is an intensity image with a scalar value representing each pixel, therefore set
NumComponents property to 1. This tutorial pads the thumbnail image with 5 inactive lines above and below, and 10 inactive pixels on the front and back of each line.
getparamfromfrm2pix function to get useful image dimensions from the serializer object. This syntax discards the first two returned values, and keeps only the total number of pixels in the padded frame. Call the object to convert the image into a vector of pixels and a vector of control signals.
Note: This syntax runs only in R2016b or later. If you are using an earlier release, replace each call of an object with the equivalent
step syntax. For example, replace
frm2pix = visionhdl.FrameToPixels(... 'NumComponents',1,... 'VideoFormat','custom',... 'ActivePixelsPerLine',imActivePixels,... 'ActiveVideoLines',imActiveLines,... 'TotalPixelsPerLine',imActivePixels+20,... 'TotalVideoLines',imActiveLines+10,... 'StartingActiveLine',6,... 'FrontPorch',10); [~,~,numPixelsPerFrame] = getparamfromfrm2pix(frm2pix); [pixel,ctrl] = frm2pix(inputIm);
Design HDL-Compatible Model
Select an image processing object from the
visionhdl library. This tutorial uses
Construct a function containing a persistent instance of this object. The function processes a single pixel by executing one call to the object.
ctrlOut arguments of the object are structures that contain five control signals. The signals indicate the validity of each pixel and the location of each pixel in the frame.
Set the filter coefficients of the
visionhdl.ImageFilter to perform a 2×2 blur operation.
For this tutorial, you do not need to change the
LineBufferSize property of the filter object. This parameter does not affect simulation speed, so it does not need to be modified when simulating with a small test image. When choosing
LineBufferSize, select a power of two that accommodates the active line size of the largest required frame format. The default value, 2048, accommodates 1080p video format.
function [pixOut,ctrlOut] = HDLTargetedDesign(pixIn,ctrlIn) persistent filt2d if isempty(filt2d) filt2d = visionhdl.ImageFilter(... 'Coefficients',ones(2,2)/4,... 'CoefficientsDataType','Custom',... 'CustomCoefficientsDataType',numerictype(0,1,2),... 'PaddingMethod','Symmetric'); end [pixOut,ctrlOut] = filt2d(pixIn,ctrlIn); end
Preallocate the output vectors for a more efficient simulation. Then, call the function once for each pixel in the padded frame, which is represented by the
pixelOut = zeros(numPixelsPerFrame,1,'uint8'); ctrlOut = repmat(pixelcontrolstruct,numPixelsPerFrame,1); for p = 1:numPixelsPerFrame [pixelOut(p),ctrlOut(p)] = HDLTargetedDesign(pixel(p),ctrl(p)); end
Deserialize Filtered Pixel Stream
visionhdl.PixelsToFrame System object converts a pixel stream to frame-based video. Use this object to deserialize the filtered data from
visionhdl.ImageFilter. Set the image dimension properties to match the test image. Call the object to convert the output of the HDL-targeted function to a matrix.
pix2frm = visionhdl.PixelsToFrame(... 'NumComponents',1,... 'VideoFormat','custom',... 'ActivePixelsPerLine',imActivePixels,... 'ActiveVideoLines',imActiveLines,... 'TotalPixelsPerLine',imActivePixels+20); [outputIm,validIm] = pix2frm(pixelOut,ctrlOut);
imshow function to display the result of the operation.
if validIm figure imshow(outputIm,'InitialMagnification',300) title 'Output Image' end
Compare to Behavioral Model
If you have a behavioral model of the design, you can compare the output frames visually or mathematically. For filtering, you can compare
visionhdl.ImageFilter with the
imfilter (Image Processing Toolbox) function in Image Processing Toolbox™. The
imfilter function operates on the frame as a matrix and return a modified frame as a matrix. You can compare this matrix with the matrix output of the
To avoid dependency on a Image Processing Toolbox license, this tutorial does not perform a compare.
HDL Code Generation
Once your design is working in simulation, use HDL Coder™ to generate HDL code for the
HDLTargetedDesign function. See Generate HDL Code from MATLAB.