How can I find pairs of rows with the same values in certain columns?

For a (very) huge data set in form of a 350000x11 matrix I need to find pairs of rows

• with identical values in columns 1,6 and 7 and
• with a different value in column 10 (dummy variable, either 0 or 1)

and calculate the difference in column 11 for each pair matching the above mentioned conditions. Using a for loop I only managed to compare consecutive rows:

[m,n] = size(X)
for i=1:m-1
if X(i,1) == X(i+1,1) && X(i,6) == X(i+1,6) && X(i,7) == X(i+1,7) && X(i,10) ~= X(i+1,10)
    X(i,11)-X(i+1,10)
end
end

Any ideas? Thank you in advance!

Answer 1

[Andrei Bobrov]

k = diff(X);
i0 = find(all(k(:,[1,6,7]) == 0,2) & k(:,10) ~= 0);
out = permute(cat(3,X(i0,:),X(i0+1,:)),[3, 2, 1]);

add

difference_in_column_11 = k(i0,11);

[Manuel]

Thank you so much for your quick answer!

[Manuel]

Sory I've got another question. Since Matlab is not able to display "out" I tend to leave out that line of code. Do you have another proposal? Especially regarding the calculation of the "difference in column 11 for each pair matching the above mentioned conditions". Thanks, Manuel

[Andrei Bobrov]

see code afre word 'add'