How do I find all indexes of duplicate names in a table column, then compare the row values for each set of duplicates

So, I have two tables (T1 and T2) that I want to merge. Both have the same number of columns and the same column variables. But the two tables have some repeating ID's.

What I need to do is find the indexes of where the ID is repeated within tables and between the tables (e.g. MN_56_4) in the tables below

So all the instances of MN_56_4 are T1 rows 4 & 5 and T2 row 4.

In this example T1 rows 4 & 5 have the same ID but different values of X, Y & Z, but T1 row 4 & T2 row 4 are identical across all columns.

Question: How do I find all the occurances of repeated ID's, including the first occurence of it?

And then how do I compare the rows to see if any are complete duplicates across all columns, or if they just have the same ID and different values of X, Y & Z?

T1 =

ID X Y Z

___________ ___ ___ ___

{'MN_56_1'} 5.1 6.3 9.1

{'MN_56_2'} 3.6 5.7 2.3

{'MN_56_3'} 4.2 3.4 1.5

{'MN_56_4'} 5.1 2.5 3.2

{'MN_56_4'} 5.7 1.2 6.5

T2 =

ID X Y Z

___________ ___ ___ ___

{'MN_58_1'} 6.2 9.2 0.5

{'MN_58_2'} 4.8 3.2 2.1

{'MN_58_3'} 5.6 3.5 2.7

{'MN_56_4'} 5.1 2.5 3.2

{'MN_56_5'} 5.7 1.2 6.5

I can't do it manually as I am dealing with tables over 10000 rows in length.

Thank you

[the cyclist]

I can't write out a solution write now, but pseudocode to do this would be something like

• unique on the first table column -- I think you'll need the third output, which points from the unique elements, back to the original)
• histcounts to count how many times each element occurs
• ismember to check the list against the other table for membership -- There is ability to check rows

[dpb]

I'd recommend to convert the ID variables to categorical first -- be faster and you can also use the "==" operator to match specific ones -- and countcats and friends can provide some of the same information as unique, sometimes in a more useful format.

Also look at join and friends when get to the combining step...