How to deploy trained Faster RCNN object detector in Simulink

Kindly assist me on how to use my trained Faster RCNN object detector in Simulink.

I have trained a Faster RCNN object detector in matlab, however, I am having difficulty in getting resources and way point on how to use it in simulink.

I humbly need urgent response.

Answer 1

[Patel Mounika]

There is no direct way to use Faster RCNN object detector in Simulink but instead you can use coder.extrinsic to wrap the call to object detector inside a MATLAB Function block.

You can refer to below link to know more about coder.extrinsic:https://www.mathworks.com/help/fixedpoint/ref/coder.extrinsic.html

For example, the MATLAB Function block can be defined as follows:

function[bboxes,scores]= function(I)
coder.extrinsic('detect');
coder.extrinsic('evalin');
[bboxes,scores] = evalin('base','detect(detector,I)'); % detector is fasterRCNNObjectDetector and I is image.

Hope this helps.

[HAMMED OBASEKORE]

Thanks so much for the response. It came in when I almost gave up on it.

However, I actualy did that, but, I received errors when loading the detector (fasterRCNNObjectDetector) from the .mat file I saved.

function y = fcn(u)
%#codegen
coder.extrinsic('load');
coder.extrinsic('detect');

persistent detector

if(isempty(detector))
    cnnn = load('C:\Users\OBAS\Pictures\Building_Insect_Set\trained\matlab_detector_vgg19_mAP0.81%250epochs.mat');
    detector = cnnn.detector;
end

kk = zeros(50,1);
[bboxes,scores] = detect(detector,u);
for i=1:size(scores)
    kk(i)=scores(i);
end
y = kk;

%% The Errors

Attempt to extract field 'detector' from 'mxArray'.

Function 'MATLAB Function' (#98.256.260), line 10, column 16:
"cnnn"
Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Persistent variable 'detector' must be assigned before it is used. The only exception is a check using 'isempty(detector)' that can be performed prior to assignment.

Function 'MATLAB Function' (#98.319.327), line 14, column 26:
"detector"
Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
...
    

Is there anyway around these errors.

[HAMMED OBASEKORE]

Thanks Patel I eventually got it to work. Here is the correct function inside Matlab Function block in simulink

function [count, bbox ,prob] = fcn(I)
%#codegen
% ensure to initialize your trainned FasterRCNN in the workspace
% I is the Image

coder.extrinsic('evalin')
coder.extrinsic('assignin')

kk= zeros(50,1,'double');
prob = zeros(50,1,'double');
bbox = zeros(50,4,'double');                 
k=0;

assignin('base','u',I)
[bboxes, scores, cat] = evalin('base','detect(detector,u)')
N = size(scores)

k =  N(1)
count = k;
for i = 1:50
    if(~isempty(k))
        if i > k
            break;
        end
    end
    kk(i) = scores(i);
    bbox(i,:) = bboxes(i,1:4);
end
prob = kk;

Answer 2

[Arkadiy Turevskiy]

As of R2021b you can use Deep Learning Object Detector block in Computer Vision Toolbox. For mpre information please see Deep Learning in Simulink page in documentation.