How to add a high-speed test in a self-defined FAST feature detector function?

Hi all,

I want to define a FAST detector which do a high-speed test on 4 pixel first, if 3 or more past test then test all 16 pixels to find a feature points. (Without using detectFASTFeatures function.)

The orgional FAST detector is this, where white points are feature points:

Sample1_double = im2double(Sample1);
Sample1_temp = my_fast_detector_no_highS(Sample1);
Sample1_mat = Sample1_double + Sample1_temp;

function fastM = my_fast_detector_no_highS(img)
    n = 12;
    threshold = 10;
    img = rgb2gray(img);
    img_temp = zeros(756,606);
    img_temp(4:753,4:603) = img;
    fastM = zeros(750,600);
    intensityD = zeros(750,600,16);
    intensityD(:,:,1) = abs(img_temp(1:end-6,4:end-3)-img_temp(4:end-3,4:end-3));
    intensityD(:,:,5) = abs(img_temp(4:end-3,7:end)-img_temp(4:end-3,4:end-3));
    intensityD(:,:,9) = abs(img_temp(7:end,4:end-3)-img_temp(4:end-3,4:end-3));
    intensityD(:,:,13) = abs(img_temp(4:end-3,1:end-6)-img_temp(4:end-3,4:end-3));
    intensityD(:,:,2) = abs(img_temp(1:end-6,5:end-2)-img_temp(4:end-3,4:end-3));
    intensityD(:,:,3) = abs(img_temp(2:end-5,6:end-1)-img_temp(4:end-3,4:end-3));
    intensityD(:,:,4) = abs(img_temp(3:end-4,7:end)-img_temp(4:end-3,4:end-3));
    intensityD(:,:,6) = abs(img_temp(5:end-2,7:end)-img_temp(4:end-3,4:end-3));
    intensityD(:,:,7) = abs(img_temp(6:end-1,6:end-1)-img_temp(4:end-3,4:end-3));
    intensityD(:,:,8) = abs(img_temp(7:end,5:end-2)-img_temp(4:end-3,4:end-3));
    intensityD(:,:,10) = abs(img_temp(7:end,3:end-4)-img_temp(4:end-3,4:end-3));
    intensityD(:,:,11) = abs(img_temp(6:end-1,2:end-5)-img_temp(4:end-3,4:end-3));
    intensityD(:,:,12) = abs(img_temp(5:end-2,1:end-6)-img_temp(4:end-3,4:end-3));
    intensityD(:,:,14) = abs(img_temp(3:end-4,1:end-6)-img_temp(4:end-3,4:end-3));
    intensityD(:,:,15) = abs(img_temp(2:end-5,2:end-5)-img_temp(4:end-3,4:end-3));
    intensityD(:,:,16) = abs(img_temp(1:end-6,3:end-4)-img_temp(4:end-3,4:end-3));
    intensityD = intensityD > threshold;
    fastM = sum(intensityD, 3);
    fastM = fastM > n;
end

I want to add a count matrix that if high_speed_count(Xi,Yi)>2, then keep calculate intensity difference sorrund img_temp(Xi,Yi); else if high_speed_count(Xi,Yi)<=2, skip to img_temp's next element and do FAST for next pixel.

I modify the upper function as follows:

function fastM = my_fast_detector(img)
    n = 12;
    threshold = 10;
    img = rgb2gray(img);
    img_temp = zeros(756,606);
    img_temp(4:753,4:603) = img;
    fastM = zeros(750,600);
    intensityD = zeros(750,600,16);
    high_speed_count = zeros(750,600);
    intensityD(:,:,1) = abs(img_temp(1:end-6,4:end-3)-img_temp(4:end-3,4:end-3));
    if intensityD(:,:,1) > threshold
        high_speed_count(:,:) = high_speed_count(:,:) + 1;
    end
    intensityD(:,:,5) = abs(img_temp(4:end-3,7:end)-img_temp(4:end-3,4:end-3));
    if intensityD(:,:,5) > threshold
        high_speed_count(:,:) = high_speed_count(:,:) + 1;
    end
    intensityD(:,:,9) = abs(img_temp(7:end,4:end-3)-img_temp(4:end-3,4:end-3));
    if intensityD(:,:,9) > threshold
        high_speed_count(:,:) = high_speed_count(:,:) + 1;
    end
    intensityD(:,:,13) = abs(img_temp(4:end-3,1:end-6)-img_temp(4:end-3,4:end-3));
    if intensityD(:,:,13) > threshold
        high_speed_count(:,:) = high_speed_count(:,:) + 1;
    end
    if high_speed_count(:,:) > 2
        intensityD(:,:,2) = abs(img_temp(1:end-6,5:end-2)-img_temp(4:end-3,4:end-3));
        intensityD(:,:,3) = abs(img_temp(2:end-5,6:end-1)-img_temp(4:end-3,4:end-3));
        intensityD(:,:,4) = abs(img_temp(3:end-4,7:end)-img_temp(4:end-3,4:end-3));
        intensityD(:,:,6) = abs(img_temp(5:end-2,7:end)-img_temp(4:end-3,4:end-3));
        intensityD(:,:,7) = abs(img_temp(6:end-1,6:end-1)-img_temp(4:end-3,4:end-3));
        intensityD(:,:,8) = abs(img_temp(7:end,5:end-2)-img_temp(4:end-3,4:end-3));
        intensityD(:,:,10) = abs(img_temp(7:end,3:end-4)-img_temp(4:end-3,4:end-3));
        intensityD(:,:,11) = abs(img_temp(6:end-1,2:end-5)-img_temp(4:end-3,4:end-3));
        intensityD(:,:,12) = abs(img_temp(5:end-2,1:end-6)-img_temp(4:end-3,4:end-3));
        intensityD(:,:,14) = abs(img_temp(3:end-4,1:end-6)-img_temp(4:end-3,4:end-3));
        intensityD(:,:,15) = abs(img_temp(2:end-5,2:end-5)-img_temp(4:end-3,4:end-3));
        intensityD(:,:,16) = abs(img_temp(1:end-6,3:end-4)-img_temp(4:end-3,4:end-3));
        intensityD = intensityD > threshold;
        fastM = sum(intensityD, 3);
        fastM = fastM > n;
    end
end

After I apply it on origional image:

Sample1_double = im2double(Sample1);
Sample1_temp = my_fast_detector(Sample1);
Sample1_mat = Sample1_double + Sample1_temp;

it shows no change.

I don't know where is the issue.

May I ask help to fix this?

Thank you.

Answer 1

[Walter Roberson]

if intensityD(:,:,1) > threshold
    high_speed_count(:,:) = high_speed_count(:,:) + 1;
end

That code is equivalent to

if all(intensityD(:,:,1) > threshold, 'all')
    high_speed_count = high_speed_count + 1;
end

except for some differences in memory management.

That is, testing an array of data with if or while is considered to succeed only if all of the values being tested are non-zero. If there is even one location intensityD(J,K,1) <= threshold then the entire if will be considered to be false.

Each time you operate on high_speed_count you are operating on the entire matrix, adding 1 to every entry. So if it happened that one of the tests succeeded then you would be incrementing all of the matrix from 0 to 1.

What you were probably thinking you were doing was asking MATLAB to locate the places in intensityD that were > threshold, and increment the corresponding locations in high_speed_count by 1. But that is not at all what you programmed.

You should learn how to use what MATLAB calls "logical indexing" https://www.mathworks.com/company/newsletters/articles/matrix-indexing-in-matlab.html

[WENQING CAO]

I am sorry, I read through "logical indexing" and I now know the problem is about the per element operation but I still couldn't solve the problem. And I try to use stupid way by use two for-loop but still failed.

function fastM = my_fast_detector(img)
    n = 12;
    threshold = 0.1;
    img = rgb2gray(img);
    img_temp = zeros(756,606);
    img_temp(4:753,4:603) = img;
    fastM = zeros(750,600);
    intensityD = zeros(750,600,16);
    high_speed_count = zeros(750,600);
    for x = 1:750
        for y = 1:600
            intensityD(x,y,1) = abs(img_temp(x,y+3)-img_temp(x+3,y+3));
            intensityD(x,y,5) = abs(img_temp(x+3,y+6)-img_temp(x+3,y+3));
            intensityD(x,y,9) = abs(img_temp(x+6,y+3)-img_temp(x+3,y+3));
            intensityD(x,y,13) = abs(img_temp(x+3,y)-img_temp(x+3,y+3));
            if intensityD(x,y,1) > threshold
                high_speed_count(x,y) = high_speed_count(x,y) + 1;
            end
            if intensityD(x,y,5) > threshold
                high_speed_count(x,y) = high_speed_count(x,y) + 1;
            end
            if intensityD(x,y,9) > threshold
                high_speed_count(x,y) = high_speed_count(x,y) + 1;
            end
            if intensityD(x,y,13) > threshold
                high_speed_count(x,y) = high_speed_count(x,y) + 1;
            end
            if high_speed_count(x,y) > 2
                intensityD(x,y,2) = abs(img_temp(x,y+4)-img_temp(x+3,y+3));
                intensityD(x,y,3) = abs(img_temp(x+1,y+5)-img_temp(x+3,y+3));
                intensityD(x,y,4) = abs(img_temp(x+2,y+6)-img_temp(x+3,y+3));
                intensityD(x,y,6) = abs(img_temp(x+4,y+6)-img_temp(x+3,y+3));
                intensityD(x,y,7) = abs(img_temp(x+5,y+5)-img_temp(x+3,y+3));
                intensityD(x,y,8) = abs(img_temp(x+6,y+4)-img_temp(x+3,y+3));
                intensityD(x,y,10) = abs(img_temp(x+6,y+2)-img_temp(x+3,y+3));
                intensityD(x,y,11) = abs(img_temp(x+5,y+1)-img_temp(x+3,y+3));
                intensityD(x,y,12) = abs(img_temp(x+4,y)-img_temp(x+3,y+3));
                intensityD(x,y,14) = abs(img_temp(x+2,y)-img_temp(x+3,y+3));
                intensityD(x,y,15) = abs(img_temp(x+1,y+1)-img_temp(x+3,y+3));
                intensityD(x,y,16) = abs(img_temp(x,y+2)-img_temp(x+3,y+3));
                intensityD = intensityD > threshold;
                fastM = sum(intensityD, 3);
                fastM = fastM > n;
            end
        end
    end
end

Could you please help me with this?

Thank you very much.

[Walter Roberson]

function fastM = my_fast_detector(img)
    n = 12;
    threshold = 10;
    img = rgb2gray(img);
    img_temp = zeros(756,606);
    img_temp(4:753,4:603) = img;
    fastM = zeros(750,600);
    intensityD = zeros(750,600,16);
    high_speed_count = zeros(750,600);
    
    intensityD(:,:,1) = abs(img_temp(1:end-6,4:end-3)-img_temp(4:end-3,4:end-3));
    mask1 = intensityD(:,:,1) > threshold;
    high_speed_count(mask1) = high_speed_count(mask1) + 1;
    
    intensityD(:,:,5) = abs(img_temp(4:end-3,7:end)-img_temp(4:end-3,4:end-3));
    mask5 = intensityD(:,:,5) > threshold;
    high_speed_count(mask5) = high_speed_count(mask5) + 1;
    
    intensityD(:,:,9) = abs(img_temp(7:end,4:end-3)-img_temp(4:end-3,4:end-3));
    mask9 = intensityD(:,:,9) > threshold;
    high_speed_count(mask9) = high_speed_count(mask9) + 1;
    
    intensityD(:,:,13) = abs(img_temp(4:end-3,1:end-6)-img_temp(4:end-3,4:end-3));
    mask13 = intensityD(:,:,13) > threshold;
    high_speed_count(mask13) = high_speed_count(mask13) + 1;
    

    mask2 = high_speed_count > 2;
    
    T = intensityD(:,:,2);
    iD = abs(img_temp(1:end-6,5:end-2)-img_temp(4:end-3,4:end-3));
    T(mask2) = iD(mask2);
    intensityD(:,:,2) = T;

    iD = abs(img_temp(2:end-5,6:end-1)-img_temp(4:end-3,4:end-3));
    T = intensityD(:,:,3);
    T(mask2) = iD(mask2);
    intensityD(:,:,3) = T;
    
    iD = abs(img_temp(3:end-4,7:end)-img_temp(4:end-3,4:end-3));
    T = intensityD(:,:,4);
    T(mask2) = iD(mask2);
    intensityD(:,:,4) = T;
    
    iD = abs(img_temp(5:end-2,7:end)-img_temp(4:end-3,4:end-3));
    T = intensityD(:,:,6);
    T(mask2) = iD(mask2);
    intensityD(:,:,6) = T;
    
    iD = abs(img_temp(6:end-1,6:end-1)-img_temp(4:end-3,4:end-3));
    T = intensityD(:,:,7);
    T(mask2) = iD(mask2);
    intensityD(:,:,7) = T;

    iD = abs(img_temp(7:end,5:end-2)-img_temp(4:end-3,4:end-3));
    T = intensityD(:,:,8);
    T(mask2) = iD(mask2);
    intensityD(:,:,8) = T;
     
    iD = abs(img_temp(7:end,3:end-4)-img_temp(4:end-3,4:end-3));
    T = intensityD(:,:,10);
    T(mask2) = iD(mask2);
    intensityD(:,:,10) = T;

    iD = abs(img_temp(6:end-1,2:end-5)-img_temp(4:end-3,4:end-3));
    T = intensityD(:,:,11);
    T(mask2) = iD(mask2);
    intensityD(:,:,11) = T;

    iD = abs(img_temp(5:end-2,1:end-6)-img_temp(4:end-3,4:end-3));
    T = intensityD(:,:,12);
    T(mask2) = iD(mask2);
    intensityD(:,:,12) = T;

    iD = abs(img_temp(3:end-4,1:end-6)-img_temp(4:end-3,4:end-3));
    T = intensityD(:,:,14);
    T(mask2) = iD(mask2);
    intensityD(:,:,14) = T;

    iD = abs(img_temp(2:end-5,2:end-5)-img_temp(4:end-3,4:end-3));
    T = intensityD(:,:,15);
    T(mask2) = iD(mask2);
    intensityD(:,:,15) = T;

    iD = abs(img_temp(1:end-6,3:end-4)-img_temp(4:end-3,4:end-3));
    T = intensityD(:,:,16);
    T(mask2) = iD(mask2);
    intensityD(:,:,16) = T;

    
    intensityD = intensityD > threshold;
    fastM = sum(intensityD, 3);
    fastM = fastM > n;

end

[WENQING CAO]

It works!!! What a convinent function, mask block all other elements therefor only focus on current element.

Thank you so much!!!

[Walter Roberson]

It would be possible to refine the code further so that the operation such as

abs(img_temp(1:end-6,3:end-4)-img_temp(4:end-3,4:end-3))

was only done for the locations of interest, instead of the current method where it is done for all locations after which we select out the locations we want.

However, it turns out that a lot of the time, selecting out the desired locations and doing arithmetic only on them, can be slower than doing the calculation on everything and then selected the desired parts of it. This is because the logical selection operation has to be done on each of the arrays involved, giving a vector result, then the calculation is done on the vectors... which can be slower than just tossing a block of memory into a parallelized multi-threaded internal operation (that might, for example, divide up the rows to process one set of rows per core.)

[WENQING CAO]

I kind of got what you mean, in matlab, matrix operation is faster than a loop because of it's shift operation; however if run it in other language then the operation on a vertor is faster.

C-family is able to do the shift operation and faster than a loop, however, I am so afread of memory leaking and making mistake on pointers, I avoid use C-family as much as possible. ε=(´ο｀*)))