Semantic segmentation for grayscale image

How can implement semantic segmentation for gray scale image? How to train that network?

[Walter Roberson]

Yes. At worst case, repmat() the grayscale along the third dimension to make it an RGB image.

[Zara Khan]

I have also RGB images. But how to implement the concept ? Means how to train that network?

Answer 1

[Image Analyst]

Start here:

https://www.mathworks.com/help/vision/ug/semantic-segmentation-using-deep-learning.html

I'm working on a simplified version where you simply just locate your input images, and labeled images folder but it's not ready to share yet.

[Zara Khan]

Have gone through this link. Is it applicable for any image?

clc; clear all; close all;
data = load('triangleSegmentationNetwork');
net = data.net;
net.Layers;
I = imread('E:\image1.png');
[C,scores] = semanticseg(I,net,'MiniBatchSize',32);
B = labeloverlay(I, C);
figure
imshow(B);
figure
imagesc(scores)
axis square
colorbar
BW = C == 'triangle';
figure
imshow(BW)

Not working

[Image Analyst]

If the Mathworks demo isn't working then make sure your input is correct (whether color image or gray scale, and size of image, if that matters). If it is correct then call tech support and tell them the demo doesn't work.

Answer 2

[Zara Khan]

It may be that my implementation is not right. I just checked. And asked whether I am doing the right thing

[Zara Khan]

Hope you will guide me

[Walter Roberson]

Please attach your image1.png

[Zara Khan]

Few images are attched. Please guide me.

[Image Analyst]

OK, but what did tech support say when you called them?

[Zara Khan]

Walter. Please my attached images.

[Walter Roberson]

When I run with your "image 1.png" and your code, I do not see any problem. It just gives an empty final plot, which is what should be expected since there are no triangles in the input image.

[Zara Khan]

That means I can train with this network for segmenting my images

[Walter Roberson]

This is the code I used:

filename = 'https://www.mathworks.com/matlabcentral/answers/uploaded_files/1050055/image%201.png';

data = load('triangleSegmentationNetwork');
net = data.net;
net.Layers;
I = imread(filename);
[C,scores] = semanticseg(I,net,'MiniBatchSize',32);
B = labeloverlay(I, C);
figure
imshow(B);
figure
imagesc(scores)
axis square
colorbar
BW = C == 'triangle';
figure
imshow(BW)

The code ran without error. The final image was empty...

Oh wait, the final images BW are not empty. What is happening is that every location is being detected as triangle, so the output is pure white (all locations match.)

[Zara Khan]

This is what I have done. Am I doing the right thing. I am not able to understand this properly. How I will say that it is segmented

[Walter Roberson]

In the below example, everything that shows up black is labeled 'background'.

The implication is that the triangle segmentation network is not very specific at all.

Your sign-language images are being segmented... but the segmentation is saying that everything in the image is part of some triangle.

filename = 'baby.jpg';

data = load('triangleSegmentationNetwork');
net = data.net;
net.Layers;
I = imread(filename);
[C,scores] = semanticseg(I,net,'MiniBatchSize',32);
B = labeloverlay(I, C);
figure
imshow(B);

figure
imagesc(scores)
axis square
colorbar

BW = C == 'triangle';
figure
imshow(BW)

[Zara Khan]

In the above example I can see the triangle. But in case of sign languages I am only getting white rectangle region. Apart from using this triangle method is there any way for semantic segmentation? How can only segment hand signs from background in this case?

[Walter Roberson]

@Zara Khan

Perhaps you should be training your own network; see https://www.mathworks.com/help/deeplearning/ug/create-simple-semantic-segmentation-network-in-deep-network-designer.html

The pre-trained Triangle Semantic Segmentation Network might simply be too weak for your purposes, especially since your sample sign images are not very triangular.