Extract information from Label images labeled with Image Labeler and use it in semantic segmentation deep learning

Hi!

I want to generate a semantic segmentation using deep learning to recognize the cown trees from a drone image. I think that I have not so clear the steps that I should follow. Could you help me please?

I have understood that to use or do a semantic segmentation using deep learning I must label the images first.

So...

1. Use Image Labeler to label the images. I'm not sure if is better use pixels label or square, Do they have differences?

2. I have labeled my image manually. For now I have 3 labels, soil, weeds and trees. Which would be the next step? I don't understand so good how I have to store the data and How is the data processed when I have to start the semantic segmentation.

• I should save the image and start with other image to create a long data labeled images?
• I should export the labels to workspace or a file?

3. I had thought manually label a few images, after that train a simple model and use this model to train more images and detect if the images are processing well. I'm not sure if I should create a new algorithm to label my images or is better do that process manually, I think that build a new algorithm it would be difficult for me.

4. How many images Do I have to process and label to obtain a simple model?

I tried explain me as well as possible, I know that are a lot of question but There are some doubtes that don't let me continue with my mork.

Thanks for your help and time!

I attach an ''labeled'' image from my work.

Answer 1

[Raunak Gupta]

Hi,

Let me clear the specific doubts.

1. I am assuming the square you are mentioning must be rectangle. Since rectangle is used for drawing bounding boxes in no way, it can be used to store pixel label data. So, Pixel label is the only way to create labels for semantic segmentation workflows.
2. For getting better results I think you require proper amount of data so that examples of soil, weed and tree are present in data. So, I suggest you go with some 30-40 images at first and see what the accuracy is, if it is too low you may need more labeled data.
3. Since essentially the end goal is to do semantic segmentation, after you get good result with few images you can always get training data from the model in terms output of a test image. So, by passing a test image, its labels will become the new image and label pair which can be used for further training.
4. That’s subjective and depends upon what all parameters are chosen in trainNetwork.

You may look at some resources to get started with semantic segmentation. These explain full workflows.

Semantic Segmentation Using Deep Learning

Getting Started with Semantic Segmentation Using Deep Learning

[Neus Alcon]

Many Thanks Raunak!

I have some doubts more... I was thinking to build by myself a network but I saw that there are pre trained networks.

I have understood that these pre trained networks are ables to recognize specific objects, I think that at the beginning these pre trained networks are not able to recognize the crown tree because is not a common object in a image.

I supose that althought at the beginning the pre trained network can not detect the crown trees but It can detect other objects, If I train the network it would be able to recognize the crown trees at the end. That's true?

My problem is that I am not used to work in that so is hard try to understand all the layers that form a neural network to try to build one that make sense. I had thought choose one simple pre trained network. Do you think that It would be better to use a pre trained network for the purpose to detect trees? In that case, Would you know if one of the pre trained networks that Matlab has is good for my work?

I'm really greatful for you help!

[Raunak Gupta]

Hi,

If you want to train from scratch I think you can take a look at segnetLayers or unetLayers where you can mention the number of layers and don't need to worry about the layers that are embeded in between. The examples in the documentation of both the functions can help to get started.

If you want to use pretrained network for semantic segmentation I would suggest looking the documentation I shared Semantic Segmentation Using Deep Learning. Here, deeplabv3plusLayers essentially is a pretrained network derived from well known image classification networks. You can check for the network you want to use by changing the name value 'network' to the given options.

Note that you need to add-on pretrained networks to MATLAB first before using deeplabv3plusLayers. You need to go to MATLAB Add-Ons menu to search for the network name and add it to MATLAB.

[Neus Alcon]

Thanks! Your answer is really useful for me!

[Raunak Gupta]

Always Happy to help!

[Neus Alcon]

Sorry Raunak, I have a last question... How much time do you think that I will need to build a simple semantic segmentation? I know that is debatable, but I would like to make me an idea.

Thanks

[Raunak Gupta]

Hi,

Time as such time to train the model or time you need to invest to complete the coding. Both are debatable but the documentation clearly explain the example so it won't be much challenging. However for training and finetuning it may require significant time based on the hardware and results. I can provide these insights only.