Create Labeled bigimage from ROIs and Masks

This example shows how to create a labeled bigimage consisting of categorical labels.

There are several ways to specify categorical label data for an image. This example shows two approaches. One approach uses polygonal ROI objects that store the coordinates of the boundaries of tumor and normal tissue. The other approach uses a mask to indicate a binary segmentation of the image into tissue and background. The example combines the information in the polygon coordinates and mask representations to create a single labeled bigimage.

Create a bigimage using a modified version of image "tumor_091.tif" from the CAMELYON16 data set. The original image is a training image of a lymph node containing tumor tissue. The original image has eight resolution levels, and the finest level has resolution 53760-by-61440. The modified image has only three coarse resolution levels. The spatial referencing of the modified image has been adjusted to enforce a consistent aspect ratio and to register features at each level.

bim = bigimage('tumor_091R.tif');

Get the spatial referencing and pixel extent of the bigimage at the desired output level.

ref = bim.SpatialReferencing(bim.FinestResolutionLevel);
pixelExtent = [ref.PixelExtentInWorldX,ref.PixelExtentInWorldY];

Load Label Data

The CAMELYON16 data set provides labels of tumor and normal regions as a set of coordinates specifying manually annotated region boundaries with respect to the finest resolution level. When pixels exist within the boundaries of both a normal region and a tumor region, the correct label for those pixels is normal tissue.

Load label data for the big image. This example uses a modified version of labels of the "tumor_091.tif" image from the CAMELYON16 data set. The original labels are stored in XML format. The modified labels are resampled and saved as MAT files.

roiPoints = load('labelledROIs.mat')

roiPoints = struct with fields:
       cancerRegions: {[344×2 double]  [53×2 double]  [539×2 double]  [247×2 double]  [37×2 double]  [161×2 double]}
    nonCancerRegions: {[46×2 double]}

Create polygonal ROI objects that store the coordinates of the tumor boundaries and normal tissue boundaries.

tumorPolys = cellfun(@(position) images.roi.Polygon( ...
    'Position',position,'Visible','on','Color','r'), ...
    roiPoints.cancerRegions);
normalPolys = cellfun(@(position) images.roi.Polygon( ...
    'Position',position,'Visible','on','Color','g'), ...
    roiPoints.nonCancerRegions);

Represent Labels as ROI Objects

Display the image overlaid with the annotated ROIs. The ROIs have the same coordinate system as the image, so changing the resolution levels of the displayed image still renders the ROIs accurately.

figure
h = bigimageshow(bim);
set(tumorPolys,'Parent',gca);
set(normalPolys,'Parent',gca);
title(['Resolution Level:' num2str(h.ResolutionLevel)]);

Zoom in to one ROI.

xlim([3940 4290])
ylim([2680 3010])
title(['Resolution Level:' num2str(h.ResolutionLevel)]);

Represent Labels as Masks

Create a mask at the coarsest resolution level for the stained region, which includes both tumor and normal tissue. The mask is 1 (true) for pixels whose grayscale value is less than 130. Fill small holes in the mask by performing morphological closing using the bwmorph function.

tissueMask = apply(bim,bim.CoarsestResolutionLevel, ...
    @(im)bwmorph(rgb2gray(im)<130,'close'));
bigimageshow(tissueMask);

Create Combined Labeled bigimage

Combine the information in the polygon coordinates and mask representations to create a single labeled bigmage.

To store the labeled image, create a writeable bigimage with data type categorical. Specify the required class names and the corresponding numeric pixel label ID values. Assign the label 0 to the 'Background' class. Assign a value to the 'UndefinedID' label that does not equal any of the pixel label IDs.

blockSize = [512 512];
bLabeled = bigimage(ref,1,'categorical', ...
    'Classes',["Background","Normal","Tumor"], ...
    'PixelLabelIDs',[0 1 2], ...    
    'UndefinedID',255, ...
    'BlockSize',blockSize);

Loop through each output bigimage, one block at a time. Determine the label of each block, then set the pixel data of the block accordingly.

To determine the label of each block, start with the mask of all tissue. Pixel values of 0 in the mask correspond to background, which matches the pixel label ID of the 'Background' class. Pixel values of 1 in the mask correspond to all tissue, which matches the pixel label ID of the 'Normal' class. Convert the polygon coordinates of tumor tissue to a mask by using the poly2mask function, then replace those pixels with the pixel label ID of the 'Tumor' class, 2.

If you have Parallel Computing Toolbox™, then you can run the loop in parallel by replacing the for statement with a parfor statement.

for cStart = 1:bLabeled.BlockSize(2):ref.ImageSize(2)
    for rStart = 1:bLabeled.BlockSize(1):ref.ImageSize(1)
        % Find the center of top left pixel of the block in world units
        xyStart = [cStart, rStart].*pixelExtent;
        
        % Clamp block edges to the border of the image
        cEnd = min(cStart + bLabeled.BlockSize(2)-1,ref.ImageSize(2));
        rEnd = min(rStart + bLabeled.BlockSize(1)-1,ref.ImageSize(1));
        
        % Find the center of bottom right pixel of the block in world units
        xyEnd = [cEnd rEnd].*pixelExtent;
        
        % Calculate the block size
        bsize = [rEnd cEnd] - [rStart cStart] + 1;
        
        % Create a grid encompassing all pixels in the block.
        [xgrid,ygrid] = meshgrid(xyStart(1):ref.PixelExtentInWorldX:xyEnd(1), ...
            xyStart(2):ref.PixelExtentInWorldY:xyEnd(2));
        
        % Get the mask of all tissue in the block
        blockNumeric = getRegion(tissueMask,1,xyStart,xyEnd);        
        % This tissue mask was created at a coarse level, so scale it up
        blockNumeric = imresize(blockNumeric,bsize);

        % Find the pixel coordinates of healthy tissue then convert the
        % polygon coordinates to a mask.
        tmask = false(bsize);
        for ind = 1:numel(roiPoints.cancerRegions)
            vertices = roiPoints.cancerRegions{ind};
            % Transform coordinates to local block pixel locations
            vertices = (vertices-xyStart)./pixelExtent+1;
            isTumor = poly2mask(vertices(:,1),vertices(:,2), ...
                bsize(1), bsize(2));
            tmask = tmask|isTumor;
        end
        
        % Some healthy tissue ROIs are enclosed within a tumor ROI. Find
        % the pixel coordinates of healthy tissue then convert the polygon
        % coordinates to a mask.
        for ind = 1:numel(roiPoints.nonCancerRegions)
            vertices = roiPoints.nonCancerRegions{ind};
            % Transform coordinates to local block pixel locations
            vertices = (vertices-xyStart)./pixelExtent+1;
            isHealthy = poly2mask(vertices(:,1),vertices(:,2), ...
                bsize(1),bsize(2));
            tmask = tmask & ~isHealthy;
        end        
        
        % Set the value of pixels in tumor regions as the corresponding
        % pixel label ID, |2|.
        blockNumeric = uint8(blockNumeric);
        blockNumeric(tmask) = 2;
        
        % Create a categorical image from the block data.
        blockCategorical = categorical(blockNumeric,...
            bLabeled.PixelLabelIDs,bLabeled.Classes);
        
        % Set the block of the categorical |bigimage| as the categorical
        % block image.
        setBlock(bLabeled,1,xyStart,blockCategorical);
    end
end

Display Overlay of Labels and Original Data

Display the image data, then display the labeled image data in the same axes. The three labels (normal, tumor, and background) appear in three different colors. Make the labels partially transparent so that you can distinguish the image content underneath.

hbim = bigimageshow(bim);
hla = axes;
hbl  = bigimageshow(bLabeled,'Parent',hla);
hbl.AlphaData = 0.7;
hla.Visible = 'off';

Zoom in to a ROI. Increase the label transparency so that you can more clearly distinguish the image content underneath.

linkaxes(findall(gcf,'Type','axes'));
xlim([3940 4290])
ylim([2680 3010])
hbl.AlphaData = 0.3;

Save Labels for Use Later

For most data sets, you can create labels once and then reuse the labels for multiple training sessions. The labeled big image, bLabeled, is backed by temporary files that do not exist across MATLAB® sessions. To reuse the labels in a different session of MATLAB, write bLabeled to a persistent location.

imageDir = 'Labels';
if exist(imageDir,'dir')
    rmdir('Labels','s');
end
labelDir = fullfile(imageDir,'labelled');
write(bLabeled,labelDir);

In a fresh session of MATLAB, you can reload the labeled bigimage by creating a new bigimage. When loading a labeled bigimage, you must also specify 'Classes', 'PixelLabelIDs', and 'UndefinedID'.

bLabeled = bigimage(labelDir, ...
    'PixelLabelIDs',[ 0 1 2],'Classes',["Background","Normal","Tumor"], ...
    'UndefinedID',255);