Identify the peers within a firm

Dear all:

I am working on a panel dataset, which contains worker ID, firm ID, and year. Each worker works for one firm only in each year - so it is a worker-year panel. I hope to create a matrix that shows whether worker j is worker i's coworker. Ideally, I hope to create a matrix as follows:

where and 1 indicates worker i's coworker's worker index in year t.

I am wondering if anyone knows how to have a quick algorithm instead looping over the worker's index (because there are thousands of workers in the data).

Thank you very much and I look forward to hearing from you!

Best,

Long

[KALYAN ACHARJYA]

Can you elaborate more (Maths)?

[Long Hong]

Hi Kalyan, thanks for asking! An easy example is as follows (two firms and one period).

IDw IDf

1 1

2 1

3 1

4 2

5 2

Then and . Hope it helps.

Answer 1

[Image Analyst]

You can simply use the firm vector and put it into pdist2(). Any distance that is zero means those two workers are in the same firm

firm2 = [firms(:), firms(:)];
distances = pdist2(firm2, firm2);

For example, if there is a zero at row 30 and column 142 of distances, it means that worker #30 and worker #142 work in the same firm.

[Long Hong]

Hi Image Analyst:

Thank you so much! It actually works very well! May I ask a quick follow-up question?

Starting from your code, assuming I call distances "C", I could do the following to get the C mentioned in my original post.

C(C>0) = (NaN);
C(C==0) = (1);
C(isnan(C)) = (0);

Hope I did not make mistakes. Thank you again for your reply.

Best, Long

[Image Analyst]

Does this help:

numEmployees = 10;
numFirms = 3;
firms = randi(numFirms, numEmployees, 1)
firm2 = [firms(:), firms(:)]
distances = pdist2(firm2, firm2) == 0
% Print out what employees work in what firm together
for k = 1 : numEmployees
	thisRow = distances(k, :);
	thisRow(1:k) = 0;
	otherEmployees = find(thisRow);
	if isempty(otherEmployees)
		continue;
	end
	for k2 = 1 : length(otherEmployees)
		fprintf('Employee %d works with employee %d in firm %d.\n',...
			k, otherEmployees(k2), firms(k));
	end
end

firms =

3

3

3

2

1

2

1

1

3

1

firm2 =

3 3

3 3

3 3

2 2

1 1

2 2

1 1

1 1

3 3

1 1

distances =

10×10 logical array

1 1 1 0 0 0 0 0 1 0

1 1 1 0 0 0 0 0 1 0

1 1 1 0 0 0 0 0 1 0

0 0 0 1 0 1 0 0 0 0

0 0 0 0 1 0 1 1 0 1

0 0 0 1 0 1 0 0 0 0

0 0 0 0 1 0 1 1 0 1

0 0 0 0 1 0 1 1 0 1

1 1 1 0 0 0 0 0 1 0

0 0 0 0 1 0 1 1 0 1

Employee 1 works with employee 2 in firm 3.

Employee 1 works with employee 3 in firm 3.

Employee 1 works with employee 9 in firm 3.

Employee 2 works with employee 3 in firm 3.

Employee 2 works with employee 9 in firm 3.

Employee 3 works with employee 9 in firm 3.

Employee 4 works with employee 6 in firm 2.

Employee 5 works with employee 7 in firm 1.

Employee 5 works with employee 8 in firm 1.

Employee 5 works with employee 10 in firm 1.

Employee 7 works with employee 8 in firm 1.

Employee 7 works with employee 10 in firm 1.

Employee 8 works with employee 10 in firm 1.

[Long Hong]

This is clever - Thank you!

[Long Hong]

Hi Image Analyst:

Thank you for the helpful insight. I hope I did not bother you too much. I am wondering if how you would apply this case into a panel data.

In the following example, I have created five different workers (idw), who are observed in both periods (year 1 and year 2) - therefore in total there are 10 observations.

idw = [1  1  2  2  3  3  4  4  5  5]';
idf = [1  1  1  2  1  2  1  1  2  2]';
idft = [11 12 11 22 11 22 11 12 21 22]'; %firm*year index
idft2 = [idft(:), idft(:)];
C = pdist2(idft2, idft2) == 0;

C =

10×10 logical array

1 0 1 0 1 0 1 0 0 0

0 1 0 0 0 0 0 1 0 0

1 0 1 0 1 0 1 0 0 0

0 0 0 1 0 1 0 0 0 1

1 0 1 0 1 0 1 0 0 0

0 0 0 1 0 1 0 0 0 1

1 0 1 0 1 0 1 0 0 0

0 1 0 0 0 0 0 1 0 0

0 0 0 0 0 0 0 0 1 0

0 0 0 1 0 1 0 0 0 1

However, some of the columns are actually placebo - they are by construction 0. For example, C(1,2) - I am not in the same firm in year 2 with myself in year 1.

I am wondering if there is a quick way to deal with the panel data setting. So, instead of generating a 10x10 matrix, I will have a 10x5 matrix. This will also free up a lot of space while running this program since I have thousands of workers.

Thank you very much and I look forward to hearing from you!

Best, Long

[Long Hong]

Dear @Image Analyst,

I am writing to ask a very quick follow-up question on the use of pdist2,

Using my current (huge) data, pdist2 requires 0.7 million x 0.7 million matrix , and my computer cannot handle such a big matrix. Is there any way to solve this problem (say, adding a sparse matrix option since there are basically 0 and 1 in the matrix)?

Thank you, and I look forward to hearing from you!

Best,

Long

[Image Analyst]

No idea. Give it a try.