post-estimation/testparm with xtscc regression using time and fixed effects dummies

[Victoria Consolvo]

Hello,

I am currently working with an unbalanced panel data set. We would like run some regressions that include time effects and fixed effects, with standard errors that are robust to autocorrelation, heteroscedasticity, and cross-sectional dependence. We have found that xtscc handles this framework nicely. Thus, my stata code at the moment looks something like (where timevar is a stand-in for my time variable in my data set, and panelvar is a stand-in for my panel variable):

xtscc dependentvar i.timevar i.panelvar

where i.timevar is the my set of time dummies, andi.panelvar is my set of fixed effects dummies.

I would like to test whether the fixed effects are equal--in my regression output, I have a set of estimated time and fixed effects coefficients (where the first time period is taken as a "reference" and the rest of the time dummies are in reference to that time period, and the same for the fixed effects) and a constant. I would then like to use testparm i.panelvar to test whether the fixed effects (in that they are in reference to the first coefficient) are jointly equal to zero (and thus, when not in reference to the first coefficient, are equal to each other).

Using testparm i.panelvar after a standard regress command gives me this test as I expect. However, I think after using testparm i.panelvar after xtscc gives me degrees of freedom in my statistic that correspond to both the time and fixed effects dummies, even though I specified only to test the fixed effects dummies for equality (i.e. F(df1, df2) rather than F(df2,df2)).

I know I can run xtscc i.timevar, fe to capture fixed and time effects as well, but then I'm not sure how to test for equality of the fixed effects in the way I can explicitly on the individual fixed effects dummies from the previous regression. Any tips on how to run a regression with fixed and time effects using xtscc, generating output such that I can test for equality of just the fixed effects to each other?
Thank you for any and all advice!

[Andrew Musau]

The following paper by Cameron and Miller provides an explanation of why it is not possible to perform this test of joint significance with cluster robust standard errors (if the number of dummies exceeds the number of regressors) or in your case, spatial robust (e.g., Driscoll and Kraay) standard errors. See pages 14-15.

[Victoria Consolvo]

Thanks very much for your response Andrew--this information is very helpful. I have looked over the paper you sent and have a few follow-up questions and details:

• I am working with a panel of approximately T = 75, and n = 65. It's unbalanced, but each n participates in each T about 45 times on average in our specific sample. Thus, when running a regression with time and fixed effects, I have T-1 time dummies, and n-1 fixed effect dummies.
• The issue you raise that I think is directly relevant here is, "And a test that 𝛼1, . . . , 𝛼𝐺 are jointly statistically significant is a test of 𝐺 − 1 restrictions (since the intercept or one of the fixed effects needs to be dropped). So even if the cluster-specific fixed effects are consistently estimated (i.e., if 𝑁𝑔 → ∞), it is not possible to perform this test if 𝐾 < 𝐺 − 1, which is often the case."
• In our case, K here I think is the number of regressors other than the fixed effects; so, K = T-1 = about 74 in one of our cases. And G - 1 = about 64. So K > G -1 for our purposes.
• In light of this, should there still be an issue when trying to run a joint significance test when we have the Driscoll-Kraay standard errors from xtscc? We have been running into the problem where the F statistic that is reported at the top of the regression output is exactly the same as the F stat reported when we use testparm, although the degrees of freedom in each of those tests changes, and I am not sure what could be causing this.
• I have considered the possibility of using reghdfe, absorbing the time effects and using the reported F statistic at the top of the output (which seems to test the joint significance of the fixed effects that I do not absorb, but I am not sure if it is testing whether the G-1 dummies are jointly equal to 0, or if the G fixed effects coefficients are equal to each other). Also, I am having some trouble modifying the syntax to include the dkraay vce option for the standard errors, which is what I would use with reghdfe in lieu of using xtscc directly.

Any further advice or tips are very much appreciated, and thanks again for your response.

[Andrew Musau]

In light of this, should there still be an issue when trying to run a joint significance test when we have the Driscoll-Kraay standard errors from xtscc?

No, and that is why you are able to get a result after the testparm command.

We have been running into the problem where the F statistic that is reported at the top of the regression output is exactly the same as the F stat reported when we use testparm, although the degrees of freedom in each of those tests changes, and I am not sure what could be causing this.

Have you checked whether the difference is very small to see? I would first compare the value of e(F) from

ereturn list

to that of r(F) after running testparm from

return list

[Victoria Consolvo]

Andrew--thanks again for your insight. Great question.
Running three variations of this regression:

reghdfe depvar i.person, absorb(time) old vce(robust, dkraay(4))

ivreghdfe depvar i.person, absorb(time) dkraay(4)

and

xtscc depvar i.time i.person,  lag(4)

with very small variations in the standard errors (not sure the source of the variation here), I can run

testparm i.person

and the F-statistic reported in the overall regression output is the same as that reported following the above command, and this holds in all three variations of syntax. The degrees of freedom in the F statistic correspond to the number of n-1 and T-1, so it seems like it's testing the equality of the fixed effects with the time effects. Is there some post-estimation routine I can perform in Stata which will test the joint significant of just the fixed effects, incorporating the spatially robust standard errors that I just estimated?

[Andrew Musau]

I do not think that there is any anomaly with the test. The results are what they are. However, you have not responded to #4. When you say "same", what do you mean? Are they the same up to 8 decimal places if you compare the results from ereturn list and return list?

testparm i.person

To be clear, this is a test that all person dummies are jointly equal to zero, or in other words, each is equal to zero, i.e., 1.person= 2.person=...=N.person=0. It is an exclusion test because if we find this to hold, we can exclude these dummies from the regression.

[Victoria Consolvo]

Thanks so much for your responses and for catching that--my apologies for not addressing that part of your reply. When I ran the code you suggested in #4, I indeed got the exact same F-statistics from both the overall regression output, as well as the testparm i.person command.

And yes, because one of the people is being used as a reference person as seems to be standard, the test of whether 2.person=...N.person=0 is what we're after, since the remaining dummies are in reference to 1.person.

I suppose my question concerns getting at a way to find out if fixed effects are equal--in particular, a way that makes it clear that I am testing only the fixed effects against each other, and not the time effects.

[Andrew Musau]

I think I finally understand what you are trying to express. Note that with clustered standard errors, the F-test denominator degrees of freedom is equal to the number of clusters less one. What you assume by clustering is that observations are not independent within clusters, but are between clusters. Therefore, the effective number of observations is the number of clusters. In the presence of strata, the denominator degrees of freedom is the number of clusters less the number of strata. In the regression, the numerator degrees of freedom is the number of estimated coefficients excluding the constant term. For the test, the numerator degrees of freedom is the number of coefficients being tested. Therefore, if you ran the following, you end up with the same results because your test is exactly what the F-test in the regression is reporting.

webuse grunfeld
xtscc invest i.company
testparm i.company

However, doing this does not make any sense to me. You should do this in the context of your full model.

[Victoria Consolvo]

Thanks for all of your help Andrew with this topic. Much appreciated.