imageDatastore for volumetric images

I want to use the imageDatastore command to prepare the training set for training a volumetric convolutional neural network-based semantic segmentation model. I followed the instructions given on the MATLAB webpage below and provided my code with multilayered Tif files representing the input images and the labled input images (ground truth).

https://www.mathworks.com/help/vision/ref/unetlayers.html

However, it seems imageDatastore just reads one of the layers (slices) and cannot process the volumetric images. Does anyone know how we should use this command for the volumetric image segmentation tasks? Or how can we prepare the training set for training a volumetric sementic segmentation model such as Unet 3D (https://www.mathworks.com/help/vision/ref/unet3dlayers.html)? Many thanks in advance.

[NAVNEET NAYAN]

Can you please provide the details of the images like how did you save these images and also about the lines of code that you are having a problem?

[Memo Remo]

Dear Navneet,

Thanks for the reply.

I converted the images to a volume tif file using a MATLAB code that uses the imwrite command with the "append" writemode. I named this file as "Train_Vol".

Then I tried to use the code below to train my model:

-----------------------------------------------------------------------

Train_imds = blockedImage(Train_Tif_Dir);

classNames = ["L","BT"];

labelIDs = [255 0];

Train_pxds = pixelLabelDatastore(Train_LebelsDir,classNames,labelIDs);

Train_ds = combine(Train_imds,Train_pxds);

net = trainNetwork(Train_ds,lgraph)

----------------------------------------------------------------------

lgraph is the model architecture, and Train_LebelsDir is the directory in which I stored the volumetric tif file that contains labels for each training images in the Train_Vol tif file.

Answer 1

[Ashish Uthama]

blockedImage cannot convert the slices to a volumetric block on its own unfortunately! blockedImage is useful when you want to break one 'unit' into multiple logical sub-units (i.e one file into many constituting blocks). So its not useful for your workflow at this moment (especially, since it looks like your 'unit' of data will easily fit in memory).

Does this help?

You would replace the first argument of the constructors for imageDatastore and pixellabelDatastore with the list of files you have, their order ought to match to ensure images and labels are paired correctly!.

% Image data
im = uint8(magic(10));
imwrite(im,'vol.tif')
imwrite(im,'vol.tif','WriteMode','append')
imds = imageDatastore(["vol.tif"],'ReadFcn',@tiffreadVolume);
v = read(imds);
size(v)
% ans =
%    10    10     2
% Label data
labels = im>50;
imwrite(labels,'labels.tif')
imwrite(labels,'labels.tif','WriteMode','append')
lds = pixelLabelDatastore(["labels.tif"],["bg", "fg"], [0 1], 'ReadFcn',@tiffreadVolume);
% Combined data, where one read gives a data,label set:
cds = combine(imds, lds);
d = read(cds)
% d{1} will be image, d{2} will be corresponding labels. 

[Memo Remo]

Yep! It works! Thank you so much, Ashish!

P.S. MATLAB forum doesn't let me accept your answer because it is posted under Rylan's answer. I think you should copy it into a new answer.

[Memo Remo]

Dear @Ashish Uthama

When I want to train the model using this method, I get the following error. I can't understand what is missing here. Do you know how I may check for the error source?

Error using trainNetwork
Invalid or deleted object.

Error in UNet_Mine_V2_CW21_001RICA_TissueNice (line 916)
net = trainNetwork(Train_ds,lgraph,options)

Caused by:
    Error using indexing
    Invalid or deleted object.

Answer 2

[Rylan]

Hi Memo,

The blockedImage object (in Image Processing Toolbox) has documentation that mentions volumes within images:

A blockedImage object is an image made from discrete blocks. Use blocked images when an image or volume is too large to fit into memory. With a blocked image, you can perform processing without running out of memory.

See https://www.mathworks.com/help/images/ref/blockedimage.html

There is an associated datastore which can be used to work with blockedImage called blockedImageDatastore.

Here is a list of examples which use blockedImageDatastore: https://www.mathworks.com/help/images/examples.html?category=large-image-files .

I hope this helps!

Rylan

[Memo Remo]

Dear Rylan,

Thanks for your help. I tried the "blockedImage" command but I am getting the following error message:

---------------------------------------------------------------------------------------------------

Error using blockedImage>pickReadAdapter

Unable to find a suitable Adapter to read E:\MATLAB\LP\CW21_001_RICA -

3D\UNET_CW21_001_Rica\UNET_CW21_001_Rica\Tissue_HL_Tif.

---------------------------------------------------------------------------------------------------

This process of using these datastore commands seems a little bit complicated. I rather prefer to store my training images in a 3D array variable as (XTrain) and the correspoding labels in another 3D array named "YTrain". This way I should be able to use "trainNetwork(XTrain,YTrain,lgraph,options)" to train my model.

However, when I tried this method I got another error message saying that the YTrain should be vector of categorical variables. Would you please let me know what does this mean? It would be great if I can use this method. It is easier to understand.

My conjecture is that I should convert the labeled images to a tall vector. For a single 2D image this probably means that I should store each column of the labeled image into a vactor consecutively, and for 3D volume images I should repeat this process for each consecutive slice. Please let me know if my conjecture is correct. Thank you.

[Ashish Uthama]

If your data can fit into memory as an array, you can point blockedImage directly to it:

 im= rand(10,10,10);
 bim = blockedImage(im);
 bimds = blockedImageDatastore(bim,'BlockSize',[2 2 2]);
 b = read(bimds);
 b{1}

ans(:,:,1) =

0.8147    0.1576
0.9058    0.9706


ans(:,:,2) =

0.1622    0.4505
0.7943    0.0838

Backing up though, maybe you could add more information for the full workflow. You mention: "I converted the images to a volume tif file using a MATLAB" What was the original source? What is the 'unit' of training data you want to use? (one block from a volume, or the full volume?).

To train, you need a datastore where one unit/read of that gives you one unit of data you want to train. You can use imageDatastore with a custom read function and use tiffreadvolume to read in one volume at a time. If these volumes are too big, and you want to train on blocks extracted from it - then consider using blockedImageDatastore (you'll like have to write a custom adapter to handle volume file like you are using, post back and I can help with that if its something you want to try).

For labels, training expects labels to be in a datatype called categorical: https://www.mathworks.com/help/matlab/categorical-arrays.html . Are you labels in uint8? Once you create a datastore, you can use a transform call to convert uint8 to categorical labels.

This might seem like a lot - but should hopefully make more sense once its all put together.

[Memo Remo]

Dear Ashish,

I appreciate your help.

The subject I am working on is a very simple volumetric semantic segmentation task. Assume that I have five 256 by 256 greyscale images that represent consecutive parallel slices of a 3D object. I took these five images and manually marked the regions of interest in each of them (ground truth).

Now using the input slices and the labeled ones I want to train a volumetric segmentation model. To train such models we need to convert the five training slices and the five labeled slices to two volumetric images each with the dimensions of (256*256*5). Now we have the training and the labeled block.

I understood that the "blockedImage" can convert these slices to a volumetric block, but how can we use the "pixelLabelDatastore" and "combine" commands to link the pixels in the training block to their corresponding labels in the labeled block to train our segmentation model?

[Memo Remo]

I tried to the ReadFcn you mentioned. Unfortunately, it is not working. It just reads a single slice of the entire volume.

----------------------------

Train_imds = imageDatastore(Tissue_HL_Tif_Dir);

Train_imds.ReadFcn = @tiffreadVolume;

---------------------------

Where "Tissue_HL_Tif_Dir" is the location of volumetric tiff file.