Issue in recognising multiple objects in an Image

I have images of Apples as well as of Oranges, which I am using as training images. The test image is an image consisting of both apple and orange. I am using GIST descriptor for feature extraction. When I train the classifier using extracted features, it gives an output as apple or orange for the test image. I have a query, as how can I make classifier recognise both of them in the test image. I am using KNN classifier

[Image Analyst]

Attach your two images, and your code, including the training set data. http://www.mathworks.com/matlabcentral/answers/6200-tutorial-how-to-ask-a-question-on-answers-and-get-a-fast-answer

[Rohan Gupta]

Hello Sir, I have attached the folder consisting of some of the train images and one test image. The knn classifier classifies the test image as either orange or apple, I am really struggling to find a way to make classifier identify both orange and apple in an image. It would be really grateful, if you can help me. Given below is my code.

clc; clear all;

imfile = dir('C:\Users\JTS_10\Desktop\Fruits\train_imag\train\*.jpg'); imfile1 = dir('C:\Users\JTS_10\Desktop\Fruits\test_imag\test\*.jpg');

a = struct2cell(imfile); train_lists = {a(1,:)};

d = struct2cell(imfile1); test_lists = {d(1,:)};

g = zeros(length(imfile),512);

l = {'Apple';'Orange'}; train_label = [repmat(l(1),10,1);repmat(l(2),10,1)]; test_label = [repmat(l(1),2,1);repmat(l(2),2,1)];

%% Extract features from Train Images

for i = 1:length(imfile) img = strcat('C:\Users\JTS_10\Desktop\Fruits\train_imag\train\',imfile(i).name); I1 = imread(img);

    % Parameters:
    param.imageSize = [256 256]; % it works also with non-square images
    param.orientationsPerScale = [8 8 8 8];
    param.numberBlocks = 4;
    param.fc_prefilt = 4;
    [gist1, param] = LMgist(I1, '', param);
    g(i,:) = gist1;
end

%% Extract features from Test Image

    img1 = strcat('C:\Users\JTS_10\Desktop\Fruits\test_imag\test\',imfile1(i).name);
    I1 = imread(img1);
    imshow(I1);
    % Parameters:
    param.imageSize = [256 256]; % it works also with non-square images
    param.orientationsPerScale = [8 8 8 8];
    param.numberBlocks = 4;
    param.fc_prefilt = 4;
    [gist2, param] = LMgist(I1, '', param);

class = knnclassify(gist2,g,train_label);

[Rohan Gupta]

It would be really helpful if someone would help me out with this problem. I would like to know if my approach is correct or not.

[william pyae]

Hi Rohan, I'm doing a similar project as yours. Could you able to post all your matlab code in the file exchange? I would like to take references from your project. Thank you so much.

Answer 1

[Image Analyst]

Why not simply look at the color? Just convert to HSV color space, mask out the background and look at the amount of orange in the image. If there's more orange than non-orange, it's an orange.

[Rohan Gupta]

You are right Sir, I should simply look for color. But, in my single test image there will be 3 or 4 fruits. So, how will the classifier detect all fruits in the test image.

[Image Analyst]

regionprops() will tell you the hue of every single region in the image. Once you've made a determination, you can assign a string with the name of the fruit. Like

props = regionprops(binaryImage, hueImage, 'MeanIntensity');
for k = 1 : length(props)
    thisHue = props(k).MeanIntensity
    if thisHue < 0.1 % or whatever
        fruitType{k} = 'Apple'
    else
        fruitType{k} = 'Orange'
    end
end

[Rohan Gupta]

Thank-You Sir, it works

Answer 2

[Takuji Fukumoto]

Why don't you use R-CNN? http://jp.mathworks.com/help/vision/ref/trainrcnnobjectdetector.html

I think you should cut block from a whole image and slide it for recognition if you want to use that classification.

[Rohan Gupta]

Could you please elaborate. What do you mean by cutting a block from whole image?

[Takuji Fukumoto]

I mean it can work if you create 'search window'. The search window is used in some detector algorithm.

https://www.mathworks.com/help/releases/R2016b/vision/ref/vision.peopledetector-class.html

RCNN find something like object first and then use classifier.

[Rohan Gupta]

I am not suppose to use RCNN or CNN for this application. Is there any other way. By the way thank you for your valuable inputs. It would be really helpful, if you could help me out further.

The single test image will have 3 or 4 fruits, How can I detect all of them?

Features of each fruits can be extracted using color as it's feature (during training), but when I train KNN classifier and give it a test image, it will give only one output (i.e orange or apple). It won't tell me the names of all the fruits present in the image.