counTRObjectCounter

Count objects using CounTR deep learning model

Since R2025a

expand all in page

Description

Add-On Required: This feature requires the Automated Visual Inspection Library for Computer Vision Toolbox add-on.

The counTRObjectCounter object creates a Counting TRansformer (CounTR) object counter model for counting the number of objects from arbitrary semantic categories in an image, without the need for training. Using this object, you can:

  • Create a CounTR object counter model using exemplar patches. Exemplar patches are small image patches that contain examples of the objects to be counted. These patches serve as reference examples that help the model identify and count similar objects in the larger input image.

  • Create a CounTR object counter model using an exemplar image with exemplar bounding box locations from which the model extracts exemplar patches.

After creating the CounTR model using the counTRObjectCounter object, count the number of objects in an image based on the exemplar patches you specify using the countObjects object function. Compute the density map of the object count in the image, based on specified exemplar patches, using the densityMap object function.

Note

This functionality requires Deep Learning Toolbox™ and the Automated Visual Inspection Library for Computer Vision Toolbox™. You can install the Automated Visual Inspection Library for Computer Vision Toolbox from Add-On Explorer. For more information about installing add-ons, see Get and Manage Add-Ons.

Creation

Syntax

counter = counTRObjectCounter(exemplarPatches)
counter = counTRObjectCounter(exemplarImage,exemplarBBoxes)

Description

counter = counTRObjectCounter(exemplarPatches) creates a CounTR object counter model using exemplar patches containing objects to count, exemplarPatches.

counter = counTRObjectCounter(exemplarImage,exemplarBBoxes) creates a CounTR object counter model using exemplar patches of objects obtained from an exemplar image, exemplarImage, at the exemplar bounding box locations, exemplarBBoxes.

example

Input Arguments

expand all

Exemplar patches containing examples of objects of interest, specified as numPatches-by-1 cell array where each cell contains a patch of size P-by-Q-by-3. The counTRObjectCounter object resizes patches to 64-by-64-by-3 and stacks them to form an input of size 64-by-64-by-3-by-numPatches to the object counter model. numPatches is the number of patches.

Tip

To select exemplar patches, crop sections of an image that contain clear, non-overlapping instances of the objects of interest. The patches you specify should be representative of the object class, capturing key features that distinguish the objects from the background or other classes.

Exemplar image containing examples of objects of interest, specified as one of these options:

Image TypeData Format
GrayscaleH-by-W numeric matrix
BinaryH-by-W logical matrix
RGBH-by-W-by-3 numeric array

Exemplar bounding boxes containing sample objects of interest to count, specified as a numShots-by-4 numeric matrix. numShots is the number of axis-aligned rectangles. The rows of the matrix have the form [x y w h]. x and y specify the upper-left corner of the rectangle, w specifies the width of the rectangle, which is its length along the x-axis, and h specifies the height of the rectangle, which is its length along the y-axis.

Properties

expand all

This property is read-only.

Exemplar patches used by the CounTR model, represented as a 64-by-64-by-3-by-numPatches array. numPatches is the number of created patches.

You can create exemplar parches from an image using the patchextractor function.

Number of exemplar bounding boxes, represented as a positive integer.

Object Functions

countObjectsCount objects using CounTR deep learning model
densityMapDensity map of object count generated using CounTR deep learning model

Examples

collapse all

This example uses:

Open Live Script

Load an exemplar image that contains objects to count, cell nuclei, into the workspace. The exemplar image and the test image in this example are cropped from an image of lymph node cells (patient_000_node_1.tif) in the CAMELYON-17 data set.

exemplarI = imread("nuclei.png");

Define exemplar bounding boxes, which signify the locations of sample cells to count.

exemplarBoxes = ...
         [272   23   38   30
          302  193   30   24
          66   83   30   28
          108  243   38   32
          168   61   30   30];

Display the exemplar image together and the exemplar bounding boxes using the insertShape function.

annotatedImage = insertShape(exemplarI,"rectangle",exemplarBoxes,"LineWidth",4);
imshow(annotatedImage)

Create CounTR Model

Create a CounTR object counter model using the exemplar image and exemplar bounding boxes.

counter = counTRObjectCounter(exemplarI,exemplarBoxes);

Count Objects in Test Image

Load a test image containing more cell nuclei into the workspace.

I = imread("nucleiTest.png");

Display the test timage in which to count objects.

imshow(I)

Compute the number of objects in the test image using the countObjects object function.

count = countObjects(counter,I);
count
count = single

498.1617

Create Density Map of Object Count

Compute the density map of the object count in the test image using the densityMap object function.

densMap = densityMap(counter,I);

Display the object count density map overlaid over the test image using the anomalyMapOverlay function. Display the approximate object count on the overlaid image using the insertText function.

densityOverlayImage = anomalyMapOverlay(I,densMap,"Blend","equal");
fontSize = min(floor(size(densityOverlayImage,2)/30),200);
gutterWidth = fontSize*9;
densityOverlayImageWText = insertText(densityOverlayImage,[size(densityOverlayImage,2)-gutterWidth 5],...
sprintf("count=%0.2f",count),FontSize=fontSize);
imshow(densityOverlayImageWText)

Tips

  • To select exemplar patches from many images for the CountTR model creation, specify the patches to the exemplarPatches input argument. To select exemplar patches from a single image, specify the exemplar image and the object bounding boxes it contains to the exemplarImage and exemplarBBoxes input argument, respectively.

  • To count objects in a datastore of images using the countObjects object function, or compute a count density map using the densityMap object function, use exemplars in the form of patches or images and bounding boxes. For best model performance, specify exemplars to the counTRObjectCounter object that are general enough to apply to the series of images in a datastore.

  • For best model performance, use images with well-separated, non-overlapping, objects. To improve performance in images that contain overlapping or occluded objects, crop the image to include only the non-overlapping objects of interest before running the CounTR model. This ensures that CounTR counts objects only in this region of interest and provides a more accurate count for your specific target objects.

References

[1] Liu, Chang, Yujie Zhong, Andrew Zisserman, and Weidi Xie. “CounTR: Transformer-Based Generalised Visual Counting.” arXiv, June 2, 2023. https://doi.org/10.48550/arXiv.2208.13721.

[2] grand-challenge.org. “CAMELYON17 - Grand Challenge.” Accessed November 8, 2024. https://camelyon17.grand-challenge.org/Data/.

Version History

Introduced in R2025a

See Also

| | |

Topics