Testing for autocorrelation i Panel Data

[Andreas Orp]

Hi

I have a panel data set on stock returns and different variables related to the businesses from 1993 to today. After deciding my model I have tested for homoskedasticity and found that this test indicates a problem with heteroskedasticity. I have also tested for RE vs. FE, and have found that FE is the model i have to use. My question is whether i need to test for autocorrelation in the FE model or is is sufficient to only use the cluster command and cluster the companies on id. I hav read a lot about this topic, and I have gotten the impression that clustering both control for potential autocorrelation and heteroskedasticity. In addition my dependent variable is returns, which is a common way to control for autocorrelation in stock data. I have tried to use the xtserial command, but i do get a error saying that there are no observations.

Thanks in advance.

[Carlo Lazzaro]

Andreas:
welcome to the list.
If you're dealing with a large N, small T dataset and (-xtreg- is the Stata command you're going to use), -cluster()-ing standard errors on panel_id can manage both heteroskedasticity and autocorrelation.
Otherwise, please come back to the list with more details.

[Andreas Orp]

Thank you for your answer. My dataset contains data from 488 companies with monthly data from 1993 to 2016. In my regression using clustering and fixed effects i have 12 999 observations. Because of some missing data the data available for each company (months) varies.

[Jesse Wursten]

You can try the xtistest and xtqptest commands from ssc. The latter will only work on a dataset without gaps though. You can use cluster-robust inference to get accurate standard errors given your specification, though if you do find strong serial correlation, it might be a sign that a dynamic panel model is more appropriate.

[Andreas Orp]

Doing the xtistest i get an error error saying: 3300 argument out of shape. In the FE-model i get a low rho-value which i have gotten the impression that can indicate that there are no autocorrelation. Regarding the number of observations and T would xtreg be efficient? As i mentioned in the first post, the dependent variable is returns and the model does also include various momentum factors (accumulated returns the past periods), but i would like to test for autocorrelation anyways. However, all the tests i have found online have given error messages.

[Jesse Wursten]

Could you add a sample of your data using dataex? Additionally, what does Stata return when you type "xtset"?

[Andreas Orp]

Below is an example of the data. When I type xtset it says that id is strongly balanced and that the date is from 01.12.1993 to 01.09.2016, but with gaps. The delta says 1 day, i dont know if this is important to change to monthly. As you can see from the samle i miss a lot of data, and as the model includes many variables the difference between the total observations in the sample and the observations in the model is high.

input int id str47 Name int Date float Return double(PEratio MTBV) long MarketCap
1 "Name" 19724 .0765 . .71 314831
1 "Name" 19755 .3421 . .76 314831
1 "Name" 19783 .0933 . 1.02 314831
1 "Name" 19814 .0293 . 1.12 314831
1 "Name" 19844 .2748 . 1.1500000000000001 314831
1 "Name" 19875 .1019 . 1.47 314831
1 "Name" 19905 -.1586 . 1.62 314831
1 "Name" 19936 .1361 . 1.36 314831
1 "Name" 19967 -.1604 . 1.54 314831
1 "Name" 19997 -.3458 . 1.3 314831
1 "Name" 20028 -.2114 . .85 314831
1 "Name" 20058 -.1404 . .67 314831
1 "Name" 20089 -.1535 . 1.34 40837
1 "Name" 20120 -.3363 . 1.1300000000000001 40837
1 "Name" 20148 .0308 . .75 40837
1 "Name" 20179 .1923 . .78 40837
1 "Name" 20209 -.0072 . .93 40837
1 "Name" 20240 .0036 . .92 40837
1 "Name" 20270 -.2842 . .92 40837
1 "Name" 20301 -.3869 . .66 40837
1 "Name" 20332 -.041 . .4 40837
1 "Name" 20362 -.1556 . .39 40837
1 "Name" 20393 -.1478 . .33 40837
1 "Name" 20423 -.2637 . .28 40837
1 "Name" 20454 -.3306 . .21 .
1 "Name" 20485 -.3446 . .14 .
1 "Name" 20514 .375 . .09 .
1 "Name" 20545 .6818 . .12 .
1 "Name" 20575 -.0223 . .21 .
1 "Name" 20606 -.1317 . .2 .
1 "Name" 20636 -.0712 . .18 .
1 "Name" 20667 -.2339 . .16 .
1 "Name" 20698 .3342 . .13 .
2 "Name" 19724 . . . 3598356
2 "Name" 19755 . . . 3598356
2 "Name" 19783 . . . 3598356
2 "Name" 19814 . . . 3598356
2 "Name" 19844 .0195 . 1.19 3598356
2 "Name" 19875 .1985 . 1.22 3598356
2 "Name" 19905 -.1178 . 1.46 3598356
2 "Name" 19936 -.0036 . 1.29 3598356
2 "Name" 19967 .0326 . 1.28 3598356
2 "Name" 19997 -.186 . 1.32 3598356
2 "Name" 20028 -.0776 1.5 1.08 3598356
2 "Name" 20058 -.0467 1.4000000000000001 .99 3598356
2 "Name" 20089 .0392 1.3 .81 3922322
2 "Name" 20120 .1557 1.3 .85 3922322
2 "Name" 20148 -.102 8.5 .98 3922322
2 "Name" 20179 .0818 7.6000000000000005 .88 3922322
2 "Name" 20209 -.0126 8.200000000000001 .9500000000000001 3922322
2 "Name" 20240 .2426 .8 .9400000000000001 3922322
2 "Name" 20270 -.0788 1 1.17 3922322
2 "Name" 20301 -.1896 1 1.07 3922322
2 "Name" 20332 -.0138 .5 .87 3922322
2 "Name" 20362 .107 .5 .86 3922322
2 "Name" 20393 -.0042 .6000000000000001 .9500000000000001 3922322
2 "Name" 20423 -.038 .4 .9500000000000001 3922322
2 "Name" 20454 -.0754 .4 .91 .
2 "Name" 20485 -.3525 .30000000000000004 .84 .
2 "Name" 20514 -.1495 1.6 .54 .
2 "Name" 20545 -.118 1.4000000000000001 .46 .
2 "Name" 20575 -.1527 1.2000000000000002 .41000000000000003 .
2 "Name" 20606 -.4237 1 .35000000000000003 .
2 "Name" 20636 .1 .6000000000000001 .2 .
2 "Name" 20667 -.4109 .6000000000000001 .22 .
2 "Name" 20698 .2407 .5 .13 .
3 "Name" 19724 -.0386 5 .56 5147341
3 "Name" 19755 .0502 4.800000000000001 .54 5147341
3 "Name" 19783 .0861 10.200000000000001 .56 5147341
3 "Name" 19814 -.0239 11 .61 5147341
3 "Name" 19844 .0863 10.8 .6 5147341
3 "Name" 19875 .0676 10.9 .65 5147341
3 "Name" 19905 -.0656 11.600000000000001 .6900000000000001 5147341
3 "Name" 19936 .0107 10.9 .65 5147341
3 "Name" 19967 -.0835 5.7 .65 5147341
3 "Name" 19997 .0783 5.2 .6 5147341
3 "Name" 20028 -.1321 5.6000000000000005 .65 5147341
3 "Name" 20058 .0302 4.4 .56 5147341
3 "Name" 20089 .028 4.5 .62 1761901
3 "Name" 20120 -.2124 4.7 .64 1761901
3 "Name" 20148 -.1053 3.5 .5 1761901
3 "Name" 20179 .0294 3.2 .45 1761901
3 "Name" 20209 .0679 3.2 .46 1761901
3 "Name" 20240 -.1639 3.9000000000000004 .49 1761901
3 "Name" 20270 -.048 3.3000000000000003 .41000000000000003 1761901
3 "Name" 20301 -.3277 3.1 .39 1761901
3 "Name" 20332 .0875 4 .26 1761901
3 "Name" 20362 .0115 4.3 .29 1761901
3 "Name" 20393 -.1534 4.4 .29 1761901
3 "Name" 20423 -.1376 9 .25 1761901
3 "Name" 20454 -.2101 7.7 .21 .
3 "Name" 20485 -.3448 6.1000000000000005 .17 .
3 "Name" 20514 -.0226 . .11 .
3 "Name" 20545 .2769 . .11 .
3 "Name" 20575 -.4277 . .14 .
3 "Name" 20606 -.3246 . .08 .
3 "Name" 20636 .6 . .17 .
3 "Name" 20667 .2083 . .27 .
3 "Name" 20698 .1379 . .32 .
4 "Name" 19724 .2381 . .98 936963

[Carlo Lazzaro]

Andeas:
can't you -mi- or -ipolate- the missing data?

[Jesse Wursten]

Try the following

gen month = mofd(Date)
xtset id month
drop if missing(Return, PEratio, MTBV, MarketCap)
xtreg Return PEratio MTBV MarketCap, fe
xtqptest
xthrtest
xtistest

[Andreas Orp]

I am running the xtistest now, it seems to take some time. Do I just type xtistest after the FE-regression, or do specifically test the error term? I saw a post here on the xtistest where this were done. Than you for your help, I really appreciate it!

[Jesse Wursten]

You can do both - if you type it after a regression it will predict the ue-error for you. If you have large data it's normal that it takes a long time. Do you have a lot of gaps in your data? If not, I'd consider trimming your dataset so that there's no gaps (but the timelength still varies). This would allow you to use the xtqptest, which is way faster and more robust too. Also note that the xtistest will only work if N > 2*T (roughly).

[Andreas Orp]

I have some gaps, if that means that there are missing data. By trimming, do you mean to remove all the observations where one of the variables in the model contains missing values, but that the date follows the remaining values with the same date as before?

[Jesse Wursten]

I have some gaps, if that means that there are missing data. By trimming, do you mean to remove all the observations where one of the variables in the model contains missing values, but that the date follows the remaining values with the same date as before?

Euhm, basically your panel can be three things as far as balance goes:

balanced: observations for each panel for each time period
gapped: gaps between observations, e.g. T0 is observed, as is T2, but T1 isn't for some panels
unbalanced (no gaps): no gaps. Hence if T1 and Tn are observed, so are T2 to Tn-1. However, the starting and end date can vary across panels (as can n)

The xtqptest will work with the last definition. The xtistest is the only one (apart from xtserial normally) that can handle the second.

Just a word of warning, while checking your case I detected a bug in the xtistest program, which is likely to produce incorrect results, I'll send you an updated ado file once I've fixed it.

[Andreas Orp]

Using the Date as time variable i get;
xtset id Date
panel variable: id (strongly balanced)
time variable: Date, 12/1/1993 to 9/1/2016, but with gaps
delta: 1 day

Using the approach you presented above i get:
bysort id: gen month=mofd(Dato)

. xtset id month
panel variable: id (strongly balanced)
time variable: month, 407 to 680
delta: 1 unit


Doing the xtqptest with the last approach i get an error saying that e do not include fixed effects, because these are removed by the FE-regression.

[Jesse Wursten]

The problem is xtset only looks at id and month/Date, not at the rest of the variables.

One option to see the true pattern is the following

xtreg Return PEratio MTBV MarketCap, fe
predict ue, ue
drop if missing(ue)
xtdes