"cleaning" a horizontal line removal code for object detection.

I have a code that removes horizontal lines in an image but the problem is that it removes some of the lines that are needed for object detection. Any ide on how to make my output "cleaner"?

here is my code:

A=imread('sheetmusic.jpg');
B=rgb2gray(A);

C=double(B);


for i=1:size(C,1)-2
    for j=1:size(C,2)-2
        %Sobel mask for x-direction:
        Gx=( ( 2*C(i+2,j+1)+C(i+2,j)+C(i+2,j+2) ) - ( 2*C(i,j+1)+C(i,j)+C(i,j+2) ) );
        %Sobel mask for y-direction:
        %Gy=((2*C(i+1,j+2)+C(i,j+2)+C(i+2,j+2))-(2*C(i+1,j)+C(i,j)+C(i+2,j)));
      
        %The gradient of the image
        if(Gx~=0)
            B(i,j)=abs(Gx);
        end
      
    end
end
figure,imshow(B); title('Sobel gradient');

and here is my input image:

and here is my output:

Answer 1

[Brian Hart]

Hi Janrex,

For a nice clean image like this, there's a very simple solution. Just compute a row sum; rows with staff lines have a lower sum:

tmp=sum(C,2);
figure;plot(tmp)

Now set those rows to white:

ind=find(tmp<1E5)
C(ind,:)=255;
figure;imagesc(C);axis image;

(The downsampled image above doesn't look great; looks better for real)

If the white lines through the notes cause issues, you could do some morphology to close them (dilate then erode).

[Image Analyst]

To expand on the white line removal process some more: You'd do an opening (erosion then dilation) with imopen() using a 3x1 vertical filter [1;1;1], or taller if the lines were more than one row tall.

openedImage = imopen(C, [1;1;1]);

This will expand the black vertically. But it will do it everywhere, even where we didn't want it to happen. So we need to use it only where there was a white line.

So then you'd make a mask using

[rows, columns] = size(openedImage);
mask = repmat(tmp<1E5, 1, columns)

Then you'd replace the filtered image with the opened image but only within the mask area.

C(mask) = openedImage(mask);

Note: this was just off the top of my head - I did not test it.

[Brian Hart]

Good catch; since it's a white line, it'd be the opposite operation.

[Janrex Pensader]

This code works perferecty but i still want to use the for loop with the sobel mask kernel and want to reduce the noise and add black pixels to the vertical lines of the notes do you have any suggestio?

[Janrex Pensader]

I have a follow up question. what if i have different images with different sizes, their row sum would be different and so applying a sum<1e5 in my code would be bad right?

[Image Analyst]

Correct. You'd have to adjust it for each size image.

Answer 2

[Image Analyst]

Ditch that simplistic algorithm and go for one of the more robust algorithms published here: Algorithms on Musical Notation. I'm sure you'll find something much better.

[Janrex Pensader]

my group plans to make an OMR with the CNN architecture and re are not allowed to use libraries.

[Image Analyst]

OK. Good for them. I'm sure they will learn a lot doing that. Not sure if any of the published papers I gave the link to do it that way since CNN's been around only for about 6 years or so. You might get a publication out of it if it's not been done before. Maybe you could at least upload it to the File Exchange.