Object Detection In Large Satellite Imagery Using Deep Learning

This example shows how to perform object detection on large satellite imagery using deep learning.

Overview

Object detection is a key component in many computer vision applications such as automated driving, surveillance, and tracking. For many of these applications, the size of the image data is typically smaller than 1K-by-1K pixels. Generally, images of this size do not require a size-management process. However, satellite images, which can be greater than 10K-by-10K pixels in size will usually require additional strategies.

The size of satellite imagery gives rise to several challenges. One challenge is the amount of memory needed to store and process the images. Object detector training and prediction on very large images is impractical due to GPU resource constraints.

Another challenge is the sparsity of objects within the images. There are often large regions in the image that do not contain any objects at all. Processing these areas is wasteful and often not useful for training object detectors.

A third challenge is class imbalance where one or more classes do not have the same number of samples as other classes. This can bias the performance of deep learning based object detectors towards the overrepresented classes.

This example shows how to apply several strategies to mitigate these challenges by:

• Using block processing during training and prediction to make better use of the available GPU resources.

• Automatically sample blocks of data from the large imagery to ensure that the blocks used for training contain objects of interest.

• Balance the class distribution in a training dataset created from sampled blocks.

This example first shows how to perform object detection on a large satellite image from the RarePlanes [1,2] dataset using a pretrained SSD object detector [3]. The second part of the example shows how to train a SSD object detector on the RarePlanes dataset. All the steps for object detection and training can be adapted to other large image datasets.

To learn more about the RarePlanes dataset, see the RarePlanes User Guide.

Load Pretrained Object Detector

Download a pretrained object detector. See the Train Object Detector example section for more information on training this detector.

downloadFolder = tempdir;
detector = helperDownloadObjectDetector(downloadFolder);

Load Satellite Image

Use blockedImage to load a test image from the RarePlanes dataset. The blockedImage object represents a very large image as a collection of smaller blocks which permits processing on a resource constrained system.

imageFilename = helperDownloadSampleImage(downloadFolder);
bim = blockedImage(imageFilename);

Use bigimageshow to display the image.

figure
bigimageshow(bim)

Perform Object Detection

Apply the pretrained object detector to overlapping image blocks from the large image using the blockedImage apply method. Overlapping blocks are necessary for object detection in large imagery because some objects may be clipped when a block is extracted from the image. If this is not addressed, the clipped objects may introduce detection artifacts. The helperDetectObjectsInBlock function, listed at the end of this example, addresses this by discarding detections that overlap the border area by more than 50%. The use of overlapping blocks ensures that an object clipped in one block is going to be fully visible in an adjacent block.

Specify the desired size of the blocks to process based on the detector input size. See Select Blocks for Training and Validation for more information on choosing a block size.

blockSize = detector.InputSize(1:2);

Specify the border size around the block to create overlapping blocks. Choose the border size based on the largest object size you expect for your application to ensure that the object is not clipped in at least one of the overlapping blocks. For the real portion of the RarePlanes dataset, the largest object is about 360-by-360 pixels. See Analyze Dataset Object Sizes to see how to determine object sizes in a dataset.

borderSize = [180 180];

Calculate the actual block size that the blockedImage apply object function should produce.

actualBlockSize = blockSize - 2*borderSize;

The apply object function executes a custom function for each block within the blockedImage. Define the custom function, helperDetectObjectsInBlock, as the function to execute for each block.

threshold = 0.3;
detectionFcn = @(bstruct)helperDetectObjectsInBlock(bstruct, detector, borderSize, threshold);

For faster throughput on a GPU, (at the cost of additional memory usage), specify a batch size value greater than one to have blocks concatenated into a batch of images. The exact amount of speed-up depends on how fast blocks can be read from the image versus the time it takes to process the batch of data. Empirical performance analysis is required to identify the ideal batch size each system. Reduce the batch size to prevent out-of-memory errors.

batchSize = 4;

Invoke the apply object function to run the object detector on overlapping blocks. Set PadPartialBlocks to true to ensure all the blocks have the same size. This simplifies the code in helperDetectObjectsInBlock because all the input blocks have the same size.

results = apply(bim, detectionFcn, ...
    PadPartialBlocks=true, ... 
    BlockSize=actualBlockSize,...
    BorderSize=borderSize, ...
    DisplayWaitbar=true,...
    BatchSize=batchSize);

Aggregate the detection results across all the blocks.

allBoxes  = vertcat(results.Source.bboxes);
allScores = vertcat(results.Source.scores);
allLabels = vertcat(results.Source.labels);

Display the all the detection results.

figure
bigimageshow(bim)
showShape("rectangle", allBoxes)

It is difficult to see the detections in the large image because the objects in the RarePlanes dataset are much smaller compared to the image. Set the x and y axis limits to zoom into a region with multiple detections.

figure
bigimageshow(bim)
showShape("rectangle", allBoxes)
xlim([2700 3300])
ylim([3800 4100])

The pretrained detector detects many of the airplanes but a several are missed. Many factors contribute to the overall performance of the detector such as the number of objects in the training data, the object detector configuration, as well as the hyperparameters used for training.

Detecting objects in satellite imagery is a challenging application and the RarePlanes dataset provides data you can use to explore various techniques to create a robust detector. This example shows you setup the training and prediction pipelines but does not explore other avenues to improve the detector as that requires additional empirical analysis.

The rest of the example shows how to train a SSD object detector on the real portion of the RarePlanes dataset.

Load Training Data

Create a directory to store the RarePlanes dataset.

dataFolder = fullfile(tempdir,"RarePlanes");

Go to the RarePlanes Dataset website, follow the instructions to download all the real images (~107 GB), and then uncompress the data into the folder created above. After uncompressing the data you should have the following folders:

<dataFolder>/RarePlanes/real/train

<dataFolder>/RarePlanes/real/test

Create a list of all the RGB images and their corresponding label data files from the train/PS-RGB_cog folder with matlab.io.datastore.FileSet. The data from this folder is used for training and validation.

trainingImagesFolder = fullfile(dataFolder,"real","train","PS-RGB_cog");
trainingLabelFolder = fullfile(dataFolder,"real","train","geojson_aircraft");

trainingImages = matlab.io.datastore.FileSet(trainingImagesFolder);
trainingLabels = matlab.io.datastore.FileSet(trainingLabelFolder);

The RarePlanes dataset contains ground truth for many object attributes. In this example, the object classes are created based on the "wing_type", which consists of four classes:

classes = ["delta"
    "straight"
    "swept"
    "variable swept"
    ];

Load the labels using a fileDatastore with the custom read function, helperReadGeoJSONGroundTruth, which is listed at the end of this example. helperReadGeoJSONGroundTruth parses the GeoJSON files that contain the ground truth information for each image and returns the latitude and longitude coordinates of polygon ROI labels around each plane.

labelDS = fileDatastore(trainingLabels, ReadFcn=@(filename)helperReadGeoJSONGroundTruth(filename,'wing_type'));

Prepare Data for Training

The polygon ROI label data is provided in latitude and longitude coordinates. To train an object detector, the polygon ROIs must be transformed to axis-aligned rectangle ROIs and the latitude and longitude coordinate values must be transformed to intrinsic image coordinates. The helperLatLonPolyToBoundingBox function uses georasterinfo (Mapping Toolbox) and geographicToIntrinsic (Mapping Toolbox) from the Mapping Toolbox™ to convert geographic coordinates into intrinsic image coordinates.

The conversion process requires the label and image data. Combine the label datastore with a datastore that returns the image filenames and create a datastore transform to apply the helperLatLonPolyToBoundingBox function to the combined datastore.

imageFileNameDS = arrayDatastore(trainingImages.FileInfo.Filename);
bldsTrain = combine(labelDS, imageFileNameDS);
bldsTrain = transform(bldsTrain, @(data)helperLatLonPolyToBoundingBox(data, classes));

Extract the transformed bounding boxes and labels.

boxLabels = readall(bldsTrain);
bboxes = vertcat(boxLabels{:,1});
labels = vertcat(boxLabels{:,2});

Inspect Dataset Statistics

It is important to understand the distribution of classes in a dataset as well as the size of objects. This can help you identify issues in your dataset prior to running training experiments and can often help you remedy certain data issues ahead of time.

Analyze Dataset Object Sizes

Approximate the size of each object using the diagonal of the bounding box.

diagonalLength = hypot(bboxes(:,3),bboxes(:,4));

Group object sizes by class.

G = findgroups(labels);
groupedDiagonalLength = splitapply(@(x){x},diagonalLength,G);

Visualize the distribution of object lengths for each class.

figure
numClasses = numel(classes);
for i = 1:numClasses
    len = groupedDiagonalLength{i};
    x = repelem(i,numel(len),1);
    semilogy(x,len,"o");
    hold on
end
hold off
ylabel("Diagonal box length (pixels)")

xticks(1:numClasses)
xticklabels(classes)

The object size analysis shows that, across all classes, most of the objects have roughly the same size. In the next section, the example shows how to use this information to select blocks for training.

Analyze Object Class Distribution

Count the labels in the training dataset to determine the distribution of classes in the training dataset. This checks whether or not the dataset classes are balanced.

originalDatasetCount = countlabels(labels);

Display the class distribution.

figure
histogram(Categories=originalDatasetCount.Label, BinCounts=originalDatasetCount.Count);

The class distribution analysis shows that this dataset is imbalanced. The delta and variable swept classes have significantly fewer samples than straight and swept. Class imbalance is a common challenge in many object detection applications. Common approaches to address this challenge include over or under sampling objects, data augmentation, specialized loss functions, and data synthesis. The RarePlanes dataset includes synthetic data to help balance the classes, but this example does not highlight that workflow. Instead, the Select Blocks for Training and Validation section below shows how to sample very large images to balance the class distribution in the training dataset.

Select Blocks for Training and Validation

As mentioned earlier, one challenge with processing large satellite imagery using deep learning is that the data must be processed in blocks due to GPU resource constraints. Use blockedImage to represent training images as a collection of blocks.

filenames = trainingImages.FileInfo.Filename;
bims = blockedImage(filenames);

The block size is a critical parameter for blocked-based object detector training. Select a block size based on the size of objects in the dataset such that the object and a sufficient amount of background is visible. This ensures that the object detector is trained on image blocks where the objects of interest are fully visible. Use the object data size analysis to guide the block size selection. In this dataset, using a block size of 512-by-512 pixels ensures that all the objects of interest are visible in the image blocks.

blockSize = [512 512];

With the block size defined, the next step is to specify which blocks to use from the training images. This is not a trivial task in satellite imagery because large areas within the images often do not contain any objects of interest. Therefore, naively selecting all the overlapping blocks from the training images using the selectBlockLocations function would create many image blocks with no objects, which do not provide any useful information during training. In addition, the class distribution analysis showed that the classes are imbalanced.

To find image blocks with objects for training and balance the training dataset, use balanceBoxLabels. This function samples blocks in the images from regions that contain objects and returns a predefined number of blocks. Areas of the image with underrepresented object classes are sampled at a higher frequency to help balance the class distribution. The sampling processing randomly shifts a sampling window to ensure objects are not at the same position in all the blocks. Set the number of blocks balanceBoxLabels should select based on the average number of object instances per class.

numClasses = height(originalDatasetCount);
numBlocks = mean(originalDatasetCount.Count) * numClasses;

Create a table from the boxes and labels and invoke balanceBoxLabels. In this example, blocks are selected from the highest resolution level.

boxLabelTable = table(boxLabels(:,1),boxLabels(:,2));
balancedLocationSet = balanceBoxLabels(boxLabelTable, bims, blockSize, numBlocks, Levels=1);

[==================================================] 100%
[==================================================] 100%
Elapsed time: 00:00:11
Estimated time remaining: 00:00:00
Balancing box labels complete.

Recompute the class distribution to verify that the class distribution is better.

bldsBalanced = boxLabelDatastore(boxLabelTable, balancedLocationSet);
balancedDatasetCount = countEachLabel(bldsBalanced);

Display the class distribution.

figure
histogram(Categories=balancedDatasetCount.Label, BinCounts=balancedDatasetCount.Count);

The balancing process increased the number of underrepresented classes, but there is still an imbalance due to the severity of the class imbalance. This will hinder the performance of the detector on the underrepresented classes. You can consider trying additional techniques to address the class imbalance such as collecting more data, use a data augmentation, or generate synthetic data. Using these additional techniques is beyond the scope of this example.

Because of the class imbalance, training a robust detector for all four classes is not feasible. This example combines all the classes into a single Airplane class. Use helperCombineClasses, to combine all the box labels to Airplane and rerun balanceBoxLabels to sample new block locations from the dataset. Although class balancing is not needed for a single class, balanceBoxLabels enables you to sample block locations from regions of the image where objects are present.

boxLabelTable = helperCombineClasses(boxLabelTable);
numObservations = 5000;
airplaneLocationSet = balanceBoxLabels(boxLabelTable, bims, blockSize, numObservations, Levels=1);

[==================================================] 100%
Elaps[==================================================] 100%
Elapsed time: 00:00:04
Estimated time remaining: 00:00:00
Balancing box labels complete.

bldsAirplane = boxLabelDatastore(boxLabelTable, airplaneLocationSet);

The number of observations is selected to be 5000 to ensure that the number of objects in the block location has as many objects as the original dataset. Count the number of labeled objects to verify the number of objects matches the original dataset.

countEachLabel(bldsAirplane)

ans=1×3 table
     Label      Count    ImageCount
    ________    _____    __________

    Airplane    13431       5000   

Split Blocks Into Training and Validation Sets

With a set of blocks sampled from the images, the next step is to split the set of blocks into a training set and validation sets. Create a blockedImageDatastore to load the selected block locations from the training images.

bimds = blockedImageDatastore(bims, BlockLocationSet=airplaneLocationSet);

Combine the blockedImageDatastore with the corresponding boxLabelDatastore.

ds = combine(bimds, bldsAirplane);

Shuffle the datastore prior to splitting into training and validation sets to ensure blocks from all images are included in both the training and validation sets.

ds = shuffle(ds);

Finally, split the selected blocks into a training set and validation set into an 80/20 split.

totalNumBlocks = bimds.TotalNumBlocks;
numBlocksTraining = round(totalNumBlocks*0.8);
dsTrain = subset(ds,1:numBlocksTraining);
dsVal = subset(ds,numBlocksTraining+1:totalNumBlocks);

Configure Object Detector

The SSD object detection network is composed of a feature extraction network followed by a detection subnetwork. The feature extraction network is typically a pretrained CNN. The detection subnetwork is a small CNN, compared to the feature extraction network and is composed of a few convolutional layers and layers specific to SSD.

This example uses ImageNet pretrained ResNet-50 for feature extraction from the Deep Learning Toolbox™ Model for ResNet-50 Network. Other pretrained networks such as MobileNet v2 or ResNet-18 can also be used depending on application requirements.

Load ResNet-50 and extract the layer graph. If Deep Learning Toolbox™ Model for ResNet-50 Network is not installed, the software provides a download link.

net = resnet50;
featureExtractionNet = layerGraph(net);

Specify the feature extraction layers to connect the SSD detection subnetworks. Choosing the optimal feature extraction layers depends on the size of objects in the training dataset requires empirical analysis.

layersToConnect =  ["activation_22_relu", "activation_40_relu"];

The SSD object detector uses anchor boxes. Use the estimateAnchorBoxes function to estimate anchor boxes based on the size of objects in the training data. In this example, 6 anchor boxes are estimated. Choosing the optimal number of anchor boxes requires empirical analysis.

numAnchors = 6;
anchors = estimateAnchorBoxes(bldsBalanced, numAnchors);

Sort the anchors by size and distribute them into two groups for each detection subnetwork in SSD. For more information on about anchor boxes, see Anchor Boxes for Object Detection.

area = anchors(:,1).*anchors(:,2);
[~,idx] = sort(area,"descend");
sortedAnchors = anchors(idx,:);
anchorBoxes = {sortedAnchors(1:3,:); sortedAnchors(4:6,:)};

Use ssdObjectDetector to configure a detector for the classes in RarePlanes. The SSD detection subnetworks are automatically added to the feature extraction network and the input size is set to match the size of the blocks.

detector = ssdObjectDetector(featureExtractionNet, classes, anchorBoxes, ...
    DetectionNetworkSource=layersToConnect,...
    InputSize=blockSize);

Specify Training Options

Specify the training options.

opts = trainingOptions("adam", ...
    MaxEpochs=20,...
    MiniBatchSize=batchSize,...
    InitialLearnRate=1e-3,...
    LearnRateSchedule="piecewise",...
    LearnRateDropPeriod=10,...
    LearnRateDropFactor=0.1,...
    Plots="none",...
    Shuffle="every-epoch",...
    BatchNormalizationStatistics="moving",...
    ValidationData=dsVal,...
    VerboseFrequency=1);

These training options were selected using Experiment Manager. For more information on using Experiment Manager for hyperparameter tuning, see Train Object Detectors in Experiment Manager.

Train Object Detector

Use trainSSDObjectDetector to train the object detector if the doTraining variable is true. Training takes about 8 hours and was run on a GPU with 12 GB of memory.

doTraining = false;
if doTraining
    detector = trainSSDObjectDetector(dsTrain, detector, opts);
end

Evaluate Object Detector

Evaluate the trained object detector on test images to measure the performance. Computer Vision Toolbox™ provides object detector evaluation functions to measure common metrics such as average precision (evaluateDetectionPrecision) and log-average miss rates (evaluateDetectionMissRate). For this example, use the average precision metric to evaluate performance. The average precision provides a single number that incorporates the ability of the detector to make correct classifications (precision) and the ability of the detector to find all relevant objects (recall).

Load the test set data using the helperLoadTestData function, which uses same loading procedure as shown when loading the training data.

[bimsTest, bldsTest] = helperLoadTestData(dataFolder);

Use blockedImage apply to run the object detector on all the test images with a detection threshold value of 0.01. The low threshold value generates many detections provides a more comprehensive view of detector performance across the full threshold range. By default, this example loads saved test results to allow the example to run quickly. Set the doEvaluation to true to recompute the evaluation results, if required.

doEvaluation = false;
if doEvaluation
    
    threshold = 0.01;
    testSetResults = apply(bimsTest, ...
        @(bs) helperDetectObjectsInBlock(bs, detector, borderSize, threshold), ...
        PadPartialBlocks=true, ...
        BlockSize=actualBlockSize,...
        BorderSize=borderSize, ...
        BatchSize=batchSize);

    %Gather the results from across all the images into a table.
    numTestImages = numel(testSetResults);
    allResults = table(...
        Size=[numTestImages 3], ...
        VariableNames=["Boxes", "Scores", "Labels"], ...
        VariableTypes=["cell", "cell", "cell"]);

    for i = 1:numTestImages
        allResults.Boxes{i}  = vertcat(testSetResults(i).Source.bboxes);
        allResults.Scores{i} = vertcat(testSetResults(i).Source.scores);
        allResults.Labels{i} = vertcat(testSetResults(i).Source.labels);
    end
else
    % Load test results.
    allResults = helperLoadTestResults(downloadFolder);
end

Use evaluateDetectionPrecision to compute the precision and recall metrics.

[ap, recall, precision] = evaluateDetectionPrecision(allResults, bldsTest);

Plot the precision and recall metrics for each class. The plot shows that the detector recalls about 35% of the objects in the test dataset at the specified threshold value. This highlights the challenging nature of the RarePlanes dataset. Improving the results may require adding data augmentation, more hyperparameter tuning, or trying another object detector such as YOLO v4 or Faster R-CNN.

figure
plot(recall,precision)
title("Airplane" + " (AP:" + ap + ")")
xlabel("Recall")
ylabel("Precision")
grid on

Summary

This example showed how to use blockedImage to implement block-based object detector training and prediction workflows. Block-based processing enables object detection on large images by breaking down large images into blocks of data that can be processed on resource constrained GPUs. Moreover, selecting blocks for training with balanceBoxLabels help mitigate class imbalance and helps select training blocks that contain objects of interest.

The SSD training and detection workflow shown in this example can be extended to other object detectors such as Faster R-CNN, YOLO v4, and YOLO v2 by changing the detector and training function used in the Configure Object Detector and Train Object Detector section, respectively. For more information about other object detectors, see Getting Started with Object Detection Using Deep Learning.

Supporting Functions

helperDetectObjectsInBlock

Run object detector on blocks of data supplied by the blockedImage apply method.

function bres = helperDetectObjectsInBlock(bstruct, detector, borderSize, threshold)

% Get the block of data. 
paddedBlock = bstruct.Data;

% Run the object detector.
[bboxes, scores, labels] = detect(detector, paddedBlock, Threshold = threshold);
if ~iscell(bboxes)
    bboxes = {bboxes};
    scores = {scores};
    labels = {labels};
end

% Determine the position of the valid block region (excluding the border
% area). This is needed to remove boxes that are detected in the border. 
actualBlockSize = size(paddedBlock,[1 2]) - 2*borderSize;
blockPosition = [borderSize([2 1])+1 actualBlockSize([2 1])];

% Offset to place boxes in data world coords. This offset is used to update
% the position of the boxes from the local block space to the world
% coordinates of the larger image.
offset = [1 1] - bstruct.Start(:,[2 1]);

for i = 1:numel(bboxes)

    % Remove boxes that lie in the border region. 
    [bboxes{i}, scores{i}, labels{i}] = ...
        helperRemoveDetectionsInBorderRegion(bboxes{i}, scores{i}, labels{i}, blockPosition);

    % Update box positions to be relative to the world coordinates.
    bboxes{i}(:,[1 2]) = bboxes{i}(:,[1 2]) - offset(i,:);
end

% Pack the detection results into a struct and ensure the last dimension
% equals bstruct.BatchSize by transposing the struct.
bres = struct('bboxes', bboxes, 'labels', labels, 'scores', scores)';

end

helperRemoveDetectionsInBorderRegion

This function removes detections that fall between the edge of two blocks and would have been cut in half.

function [bboxes, scores, labels] = helperRemoveDetectionsInBorderRegion(...
    bboxes, scores, labels, blockPosition)

% Use bboxcrop to find out which boxes are inside the block position. 
[~, valid] = bboxcrop(bboxes, blockPosition, OverlapThreshold=0.5);
bboxes = bboxes(valid,:);
scores = scores(valid);
labels = labels(valid);
end

helperPolygonToBoundingBox

Converts M four-sided polygons stored in a 4-by-2-by-M array to axis-aligned bounding boxes stored in an M-by-4 matrix.

function bbox = helperBboxFromPolygon(poly)
X = poly(:,1,:);
Y = poly(:,2,:);
X = squeeze(X);
Y = squeeze(Y);
xmin = min(X)';
xmax = max(X)';
ymin = min(Y)';
ymax = max(Y)';

bbox = [xmin ymin xmax-xmin ymax-ymin];
end

helperLatLonPolyToBoundingBox

Convert a polygon specified in latitude and longitude coordinates to a bounding box in pixel coordinates. This function uses georasterinfo and geographicToIntrinsic from the Mapping Toolbox™.

function out = helperLatLonPolyToBoundingBox(data, classes)
poly = data{1}{1};
imageFile = data{2};
rasterInfo = georasterinfo(imageFile);
rasterRef = rasterInfo.RasterReference;
for i = 1:size(poly,3)
    lon = poly(:,1,i);
    lat = poly(:,2,i);
    [xi,yi] = geographicToIntrinsic(rasterRef,lat,lon);
    poly(:,1,i) = xi;
    poly(:,2,i) = yi;
end

bbox = helperBboxFromPolygon(poly);

if nargin == 1
    % Use single Airplane class.
    numObjects = size(bbox,1);
    labels = repmat("Airplane",numObjects,1);
    labels = categorical(labels,"Airplane");
else
    % Use classes from input. 
    labels = categorical(data{1}{2}, classes);
end
out = {bbox, labels};
end

helperDownloadObjectDetector

Download a pretrained YOLO v4 object detector.

function detector = helperDownloadObjectDetector(folder)
url = "https://ssd.mathworks.com/supportfiles/vision/data/pretrainedSSDRarePlanes.zip";
zipFile = fullfile(folder,"pretrainedSSDRarePlanes.zip");
if ~exist(zipFile,"file")
    websave(zipFile, url);
end
matFile = fullfile(folder, "pretrainedSSDRarePlanes.mat");
if ~exist(matFile, "file")
    unzip(zipFile,folder);
end

pretrained = load(matFile);
detector = pretrained.detector;
end

helperReadGeoJSONGroundTruth

Read GeoJSON ground truth data from the RarePlanes dataset. For more information about the ground truth format, see the RarePlanes User Guide.

function [data, info] = helperReadGeoJSONGroundTruth(filename, attribute)

% Load labels from JSON file.
txt = fileread(filename);
labelsJSON = jsondecode(txt);

s = [labelsJSON.features(:).geometry];

% Concatenate coordinates into numObjects-by-5-by-2. These are the
% coordinates of the polygon used to label the planes. The coordinates are
% stored as [longitude latitude] in GeoJSON.
lonlat = cat(1,s(:).coordinates);

% Discard the last coordinate. It is the same as the first and closes the
% polygon.
lonlat(:,5,:) = [];

% Permute the coordinates to 4-by-2-by-N and then to a cell of array of
% M-by-2 matrices.
lonlat = permute(lonlat,[2 3 1]);

% Extract object property information.
info = vertcat(labelsJSON.features(:).properties);

% Extract out object label information. 
labels = string({info(:).(attribute)}');

data = {lonlat labels};
end

helperLoadTestData

Load RarePlanes test data.

function [bimsTest, bldsTest] = helperLoadTestData(dataFolder)
% Load test images.
testImagesFolder = fullfile(dataFolder,"real","test","PS-RGB_cog");
testImages = matlab.io.datastore.FileSet(testImagesFolder);
bimsTest = blockedImage(testImages);

% Load test box labels.
testLabelFolder = fullfile(dataFolder,"real","test","geojson_aircraft");
testLabels = matlab.io.datastore.FileSet(testLabelFolder);
testLabelDS = fileDatastore(testLabels, ReadFcn=@(filename)helperReadGeoJSONGroundTruth(filename,'wing_type'));
imageFileNameDS = arrayDatastore(testImages.FileInfo.Filename);
dsTest = combine(testLabelDS, imageFileNameDS);
bldsTest = transform(dsTest, @(data)helperLatLonPolyToBoundingBox(data));
end

helperLoadTestResults

Load saved test results.

function allResults = helperLoadTestResults(folder)
url = "https://ssd.mathworks.com/supportfiles/vision/data/pretrainedSSDRarePlanes.zip";
zipFile = fullfile(folder,"pretrainedSSDRarePlanes.zip");
if ~exist(zipFile,"file")
    websave(zipFile, url);
end
matFile = fullfile(folder, "pretrainedSSDRarePlanes.mat");
if ~exist(matFile, "file")
    unzip(zipFile,folder);
end

pretrained = load(matFile);
allResults = pretrained.allResults;
end

helperCombineClasses

Combine airplane classes into a single Airplane class.

function boxLabelTable = helperCombineClasses(boxLabelTable)
for i = 1:height(boxLabelTable)
    numLabels = numel(boxLabelTable{i,2}{1});
    boxLabelTable{i,2}{1} = repmat("Airplane", numLabels, 1);
end
end

helperDownloadSampleImage

Download a sample image from the RarePlanes dataset.

function filename = helperDownloadSampleImage(folder)
url = "https://ssd.mathworks.com/supportfiles/vision/data/RarePlanesSampleImage.zip";
zipFile = fullfile(folder,"RarePlanesSampleImage.zip");
if ~exist(zipFile,"file")
    websave(zipFile, url);
end
filename = fullfile(folder, "113_104001003D8DB300.tif");
if ~exist(filename, "file")
    unzip(zipFile,folder);
end
end

References

[1] J. Shermeyer, T. Hossler, A. Van Etten, D. Hogan, R. Lewis, and D. Kim, RarePlanes Dataset. In-Q-Tel - CosmiQ Works, 2020.

[2] J. Shermeyer, T. Hossler, A. Van Etten, D. Hogan, R. Lewis, and D. Kim, “RarePlanes: Synthetic Data Takes Flight,” Jun. 2020.

[3] W. Liu, E. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.Fu, and A.C. Berg. "SSD: Single Shot MultiBox Detector." European Conference on Computer Vision (ECCV), Springer Verlag, 2016