Fixed effect DDD using areg

[Peter Li]

Hi everyone, this my first post here at Statalist!

I've been browsing through posts, but couldn't find exactly what I'm looking (but the nature of my question is very similar to this post here).

I'm trying to do a triple diff model using areg, and my model includes both year and state fixed effects

My current code is:

 areg lwage i.teacher##i.treat##i.post `controls' i.year, absorb(statefip) cluster(statefip)

(1)



My questions are:

1. First of all, is my above code correct?
Or should I do this instead since the 3-way interaction is all I care about?

 areg lwage 1.teacher##1.treat##1.post `controls' i.year, absorb(statefip) cluster(statefip)

(2)

2. What I noticed is, the (2) model gives me significant interaction coefficient estimate, but the (1) model don't. Why is that?
I suspect it is because the main effects are not there in the (2) model.

3. In general, are there good reasons to simply estimate the interaction effect without the main effect if the interaction effect is all I care about?


Thank you so much!

Best,
Peter

[Clyde Schechter]

First, I can't reproduce what you are saying. When I do similar things to what you show using a data set available to me, models 1 and 2 give me the same exact output. (I'm assuming here that teacher, treat, and post are all dichotomous variables.) Perhaps what you meant in model 2 is: 1.teacher#1.treat#1.post (with single #, not double ##)?

If that's the case, suffice it to say that the model with just 1.teacher#1.treat#1.post is not a valid DID model and its coefficients cannot be interpreted as if it were. In that model, having the interaction term without the main effects, the coefficient of that interaction term is the expected difference in outcome lwage between those who are teacher = 1, treat = 1 and post = 1 vs all other observations. That is not a DID estimator. In fact, that model is almost certainly a gross mis-specification of reality because it constrains things so that teacher, treat, and post have no effects whatsoever except that the combination where all of them = 1 is different from everything else.

Finally, beware the siren song of "significance." Based on what you wrote, you are eager for a "significant" result. But the American Statistical Association now recommend abandoning the entire concept of statistical significance, a deeply flawed concept that has had very destructive effects on the quality of research (mostly because it is badly misunderstood by so many who use it). See https://www.tandfonline.com/doi/full...5.2019.1583913 for more. It is particularly pernicious to go hunting for statistically significant results, or to choose your results based on that criterion.

[Peter Li]

Clyde Schechter , thank you so much! Your explanation makes perfect sense. And thank for you for the note on "significance"!

And yes, you're correct, all three main variables are dichotomous, and the interaction in model 2 is 1.teacher#1.treat#1.post. I will make sure to specify in the future.

So going back to my first question, is my code for model (1) correct? should I use i.teacher#i.treat#i.post or i.teacher##i.treat##i.post ?

I know it is recommended to use # instead of generating manually interaction terms, but in this case, when I used i.teacher##i.treat##i.post, 1.treat gets dropped automatically by Stata (which makes sense since it is in the state FE), but 1.post is not dropped, even though i.year (or year FE) is specified in the model. Why is that?

Thank you again!

[Clyde Schechter]

When specifying regression models with interactions I nearly always use the ## notation, not the #. The reason is that in most cases, the model is not valid unless the "main" effects are included along with the interaction. By using ##, that is taken care of automatically and you cannot mistakenly leave something out. You correctly note that Stata then goes ahead and omits treat, because it is colinear with the state fixed effect. No problem with that: Stata will do that whenever it's necessary, and it will never do it when it's unnecessary. Makes it bulletproof.

As for post not being dropped even though it is colinear with the year fixed effects, if you look closely you will notice that Stata has omtited two years, rather than just the usual one. In other words, it resolves the colinearity in a different way. If you find this aesthetically unpleasant, move i.year to the front of the list of predictors and Stata will then drop post and keep all but one of the years. But it's really nothing more than a matter of taste: it will not affect the estimation of any of the effects of interest (although the year coefficients will be different.) In most cases, when Stata resolves a colinearity it does so by removing variables from the end of the list of predictors (but not including anything that was set as a panel variable in -xtset-, and not including anything in the -absorb()- option of -areg-). So you can in theory control which variable Stata omits by the order in which you write the variable list--but, again, this never has any substantive consequences: nothing outside the colinearity is affected by that, and the coefficients of the colinear variables are unidentifiable anyway. Also, for reasons known only to StataCorp, sometimes Stata chooses something different from what my general rule says it will. So I almost invariably just write the variables in the order of decreasing importance to me and let Stata break colinearities however it sees fit. If it really matters to me (almost never), then I do the omissions myself explicitly.

[Peter Li]

Clyde Schechter You're right! Stata did omit two years. Mystery solved.

This case is closed, thank you so much, Clyde!