How to apply sift to each superpixel in image ?

This implementation is to apply sift to an image that works correctly:

[Locations,Descriptors]= vl_sift(I);
Locations=Locations([2,1,3,4],:)';
Descriptors=Descriptors';
num_keypoint=size(Locations,1);
Descriptors=double(Descriptors);
Descriptors=Descriptors./repmat(NormRow(Descriptors,2),1,128);

And this is my implementation to apply sift to each superpixel, But I believe it's not right and ultimately leads to a false detection.

Locations=[];
Descriptors=[];
for p=1:num_segments
    seg_map=(segments==p); % Specify each segment
%                seg_map( ~any(seg_map,2), : ) = [];  %rows
%                seg_map( :, ~any(seg_map,1) ) = [];  %columns
%        seg_map=double(seg_map);
%        seg_map(seg_map==0) = nan;
    [yy,xx]=size(seg_map);
    sift_segments = zeros('like',grayimage);
    for i=1:yy
        for j=1:xx
            if seg_map(i,j)==0
                sift_segments(i,j)= 0;
            else
                sift_segments(i,j)=grayimage(i,j);
            end
        end
    end
    II=single(sift_segments);
%     II(II==0) = nan;
    [Loc,Des]= vl_sift(II, 'PeakThresh', Sift_Treshold(p)); 
    
    Loc=Loc([2,1,3,4],:)';    
    Locations=[Locations;Loc];    
    Des=Des';                          
    Descriptors=[Descriptors;Des];  
end
num_keypoint=size(Locations,1);
Descriptors=double(Descriptors);
Descriptors=Descriptors./repmat(NormRow(Descriptors,2),1,128);

Please pay attention to the lines that have been commented.

And the problem is that the location and descriptors extracted of each superpixel do not Correctly integrate into a matrix.

Have you got a solution?