I only want to include the observations that have observations through the whole timeframe.

I have 979588 x 12 table with US stocks from 1963 - 2011. It's a total of 6411 different stocks. I want to remove all the stocks that have not been listed the entire time. The first column is an identification number for the specific stock called permno. If sum(permno == 10000) is 588, where 10000 is the identification number, it means the stock with id 10000 has been listed the whole time. So I want to remove all the stocks that don't meet the criteria: sum(permno == 10000) is 588. How do I go about this?

[dpb]

Need more info on how the table is organized --

[Hnor]

The table is organized as follows:

Answer 1

[dpb]

NFULL=588;     % the full file number -- use variables instead of "magic numbers" in code
tSave=groupfilter(tTable,'Permnoco',@(x)numel(x)==NFULL,'Permnoco');

You didn't tell us what the variable name is for the table; substitute that in for "tTable" above.

The variable name according to the above image is 'Permnoco' not 'permno'; obviously also use the correct name inside the table.

See doc groupfilter for details on the syntax.

[Hnor]

Many thanks! That works perfect!

[dpb]

Of course it does... <VBG> Seriously, glad to help.

Explore the documentation for the table object thoroughly -- it is an extremely powerful object with all the functionality builtin and around it. rowfun, in particular with grouping variables can cut seemingly intractable problems down to trivial ones.

[Hnor]

Furthermore, we want to run AR regressions for return for up to 240 lags for each specific stock in the table. For example for Permnoco 10065 we want to first run one regression , where Return, and then one and so on for up to t-240, and then store all the betas in a timetable with columns of Permnoco and rows of dates. Do you have any idea how to go about this?

[dpb]

Well that sounds very fishy to treat independently in that manner, but it would just be a big loop to generate the output array for each set of lags.

This would have to be an m-file; couldn't be done as an anonymous function, but rowfun and the grouping variable by ID would still be the top level driver.

[Peter Perkins]

There's probably a half dozen simple-ish ways to do this, but kudos to dpd for spotlighting the relatively new groupfilter. Exactly what it is aimed at!

[dpb]

Yeah, it's indeed abrightidea® and nice addition, Peter...