Why my bounding boxes are empty?

Hello

I'm trying to run a faster rcnn model on my data. This data was created using image labelar. Matlab creates this error:

Invalid transform function defined on datastore.

The cause of the error was:

Error using vision.internal.cnn.validation.checkTrainingBoxes (line 12)

Training data from a read of the input datastore contains invalid bounding boxes. Bounding boxes must be

non-empty, fully contained within their associated image and must have positive width and height. Use

datastore transform method and remove invalid bounding boxes.

Can anyone help me?

This is my code:

clear all
close all
clc
format long g
data=load('matlab_V21_ROI_15_7_2021_slozka.mat') %matlab_V21_ROI_11_3_2021.mat
gTruth=data.gTruth
gTruth.DataSource
gTruth.LabelDefinitions

trainingDataTable=objectDetectorTrainingData(gTruth)
blds=boxLabelDatastore(trainingDataTable(:,2))
datasetCount=countEachLabel(blds)
bim=blockedImage(data.gTruth.DataSource.Source)
bls=selectBlockLocations(bim, "BlockSize",[256, 256])
bimds=blockedImageDatastore(bim, "BlockLocationSet", bls); % BlockLocationSet seznam bloků v obrazku
cds=combine(bimds,blds)
read(cds)
network = resnet50;%squeezenet; % resnet50
inputImageSize =[224 224 3]; %network.Layers(1).InputSize;
numClasses = 1;
featureLayer = 'activation_40_relu';%'fire5-concat' %'activation_40_relu';
%estimate anchor boxes
numAnchors = 3
[anchorBoxes,meanIoU] = estimateAnchorBoxes(cds,numAnchors);
anchorBoxes
meanIoU
maxNumAnchors = 3;
meanIoU = zeros([maxNumAnchors,1]);
anchorBoxes = cell(maxNumAnchors, 1);
for k = 1:maxNumAnchors
    % Estimate anchors and mean IoU.
    [anchorBoxes{k},meanIoU(k)] = estimateAnchorBoxes(cds,k);    
end
figure
plot(1:maxNumAnchors,meanIoU,'-o')
ylabel("Mean IoU")
xlabel("Number of Anchors")
title("Number of Anchors vs. Mean IoU")
% anchor boxes - pokracovani
anchorBoxes = [ 72,70; 99,96; 53,51]; %72,70; 99,96; 53,51
lgraph = fasterRCNNLayers(inputImageSize,numClasses,anchorBoxes, ...
                          network,featureLayer);
analyzeNetwork(lgraph);
options = trainingOptions ( 'sgdm' , ... 
      'MiniBatchSize' , 1, ... 
      'InitialLearnRate' , 1e-1, ... 
      'ExecutionEnvironment','cpu',...
      'MaxEpochs' , 5);
detector = trainFasterRCNNObjectDetector (cds, lgraph, options, ... 
        'NegativeOverlapRange' , [0 0.3], ... 
        'PositiveOverlapRange' , [0.6 1]);
    

[Walter Roberson]

It is not clear to me why you estimate the anchor boxes, but then overwrite them with constant locations ?

[Monika Zikmundová]

I used this code according to this example:

https://www.mathworks.com/help/vision/ug/estimate-anchor-boxes-from-training-data.html

[Image Analyst]

Aren't we going to need the .mat file you loaded? Please attach it.

[Monika Zikmundová]

Hello.

This is the link where the used image is shared.

https://u528-my.sharepoint.com/:u:/g/personal/zikmundmo_natur_cuni_cz/EWrB0QuRENlPkgx3RL85g7cB3RSRmagrTW83wYGH3jhs0Q?e=WlTNPy

[Walter Roberson]

But after you estimate the anchor boxes, you have

anchorBoxes = [ 72,70; 99,96; 53,51]; %72,70; 99,96; 53,51

which throws away the work you did estimating the anchor boxes.

[Monika Zikmundová]

What solution do you propose?

[Monika Zikmundová]

My solution:

%estimate anchor boxes

numAnchors = 3

[anchorBoxes,meanIoU] = estimateAnchorBoxes(cds,numAnchors);

anchorBoxes

meanIoU

maxNumAnchors = 3;

meanIoU = zeros([maxNumAnchors,1]);

anchorBoxes = cell(maxNumAnchors, 1);

for k = 1:maxNumAnchors

% Estimate anchors and mean IoU.

[anchorBoxes{k},meanIoU(k)] = estimateAnchorBoxes(blds,k);

end

figure

plot(1:maxNumAnchors,meanIoU,'-o')

ylabel("Mean IoU")

xlabel("Number of Anchors")

title("Number of Anchors vs. Mean IoU")

% anchor boxes - pokracovani

anchorBoxes = cell2mat(anchorBoxes(3,1))

But my problem is still the same:

Error using vision.internal.cnn.validation.checkTrainingBoxes (line 12)

Training data from a read of the input datastore contains invalid bounding boxes. Bounding boxes must be

non-empty, fully contained within their associated image and must have positive width and height. Use

datastore transform method and remove invalid bounding boxes.

Answer 1

[T.Nikhil kumar]

Hello Monika,

As per my understanding, you want to train a Faster RCNN network on your own dataset and are facing an invalid bounding box error.

The error message informs that the datastore contains inputs with invalid bounding boxes. The values of valid bounding boxes should be finite, positive, non-fractional, non-NaN and should be within the image boundary with a positive height and width.

All the invalid instances of bounding boxes in the dataset need to be deleted or modified to valid values.

You can apply the ‘transform’ function on the datastore and define your own transformation function to remove/modify such invalid bounding boxes. You can refer to the ‘@fcn’ input argument section in the following documentation to understand more about implementation of custom transformation function.

https://www.mathworks.com/help/matlab/ref/matlab.io.datastore.transform.html#mw_299e7ffc-311a-4c9d-ae67-768f942999d0

You can refer to a similar implementation of a custom dataset validation function ‘validateInputData.m’ in the ‘Object Detection using YOLOv3 network’ example attached below. (Open the example using ‘Copy Command’ to view the definition of the function)

https://www.mathworks.com/help/vision/ug/object-detection-using-yolo-v3-deep-learning.html

I also suggest you not to hardcode the ‘anchorBoxes’ value but to calculate them using the ‘estimateAnchorBoxes’ function since it may add to better accuracy to the network.

Hope this resolves your query.