Placebo test for model with fixed-effects?

[Mariela Dal Borgo]

Dears,

I have a question about model specification that I wonder if people can share their thoughts about it (by the way, apologies if this is not the place to ask this question, in that case I'll appreciate if someone can suggest me a more appropriate forum). I am estimating a model using a longitudinal dataset at the individual level. The policy, Xst, (bankruptcy protection in USD values) is implemented at the state level, since each state changes the level of protection at different times and by different amounts. I want to evaluate its effect on an outcome (stock ownership, Yist) measured at the individual level. Thus, I estimate the model using state and individual fixed-effects:

Y,ist = beta,t + beta,s + beta,i + Xs,t + state and individual controls,ist + epsilon,ist

Since I'm using a nonlinear specification (actually is a cubic spline, so it's not reflected in the equation above but I think it doesn't matter for the point I want to make), I'd like to do a placebo test to show that my results are not purely driven by the flexibility of the functional form I've chosen. I've seen that in program evaluation sometimes people use a dynamic model where they include leads and lags of the treatment variable. If the program is indeed causing the effects in the outcome, then the leads should not be significant (otherwise they will capture anticipatory effects or pre-existing trends), whereas the lags will capture short/long term effects.

I wonder if in my setting that is not program evaluation (ie my treatment is not a dummy before and after the policy was implemented), that approach would still be valid. Since fixed-effect models can be seen as a generalization of diff-in-diffs with more than 2 periods and 2 groups (http://web.mit.edu/14.771/www/emp_handout.pdf), I thought I could use a similar placebo test as the ones used in DD. Alternatively, is there a better approach to do a placebo test or is there an argument for why such kind of test shouldn't be necessary/appropriate in this context?

Well, any comments about this will be appreciated!
Thanks,

Mariela

[Jeff Wooldridge]

Mariela: Putting in one or more lead values of the policy variable is a perfectly good placebo test in such contexts. As far as lags go, in some cases I would expect there to be persistent effect. Sometimes people define dummy variables for different number of years that the policy has been in place, and then tests whether they are all equal. JW

[Mariela Dal Borgo]

Dear Jeff,

Thanks for your reply, I knew you were going to know the answer! That's good news, it's just that I wasn't sure because my policy is not a dummy variable but a variable in dollar amounts that captures the intensity of the treatment. Although as you said I can create a dummy instead for the years in which the treatment has changed.. But since the magnitude of the changes varies a lot, I'd prefer not to rely very much on that since sometimes it's purely a change to compensate for the inflation and some other times the amount more than doubles.

Just two more questions to clarify.. Would it be fine to use only one (or two) lag(s) and one (or two) lead(s) only? I don't have data on the program for many years before/after, so I may have to restrict the sample by eliminating the first/last year to add lags and leads and my panel is not very long either...

Also, I'm not sure how to interpret the coefficients on the lags. In program evaluation I've seen that people interpret them with respect to a baseline category (the situation with no program) and thus they represent the cumulative effect of the program one year after, two years after, etc. But if the treatment is defined as a continuous variable (the level of protection in USD amounts that can increase over time), is it OK to interpret them as the cumulative effect until a particular year, where the baseline will be the situation with lower protection?

Thanks a lot!

Mariela

[Jeff Wooldridge]

Hi Mariela: It was nice to see your name pop up on this forum. Sounds like a good project. I would think one or two leads would be sufficient for the placebo test. As for lags, I would interpret it as any distributed lag model. So if I were evaluating a job training program and I had hours of participation, and not just an indicator, I might use a model with a few lags of hours. Then you can see how persistent the effect is, and also estimate the long-run effect as the sum of the coefficients. If you have ignorability then you are estimating the effect of each hour. In your case, it's the effect of each dollar. You could do your analysis even without a control group if the variation in protection is sufficiently exogenous. Having zeros for some observations can only help. I hope all is well. Cheers, Jeff

[Mariela Dal Borgo]

Dear Jeff,

Thanks, this is very useful. You confirmed me what I suspected about the number of leads and I wasn't familiar with the distributed lag model, so I'll look at it too. Actually my control group are households in states where the protection didn't change or changed by a smaller amounts.
It was great to see you in this forum too and to be able to get your comments on these issues that I've been thinking for a while. Thanks again for all the help and hope everything is going well there too!

Best wishes,

Mariela

[Henrik Dalriksson]

Hi!

I am also performing a diff-in-diff and I am worried about the anticipatory effect. I don't fully understand why a lead would control for the anticipatory effect. This is because the anticipatory effect should occur before the treatment date, hence leading variables (which are forward looking) would not pick it up even if it existed. Instead, the anticipatory effect should be picked up by the lagging variable?

For example this author include a dummy which equals one precisely after the announcement and at a time closer to the treatment date.

http://faculty.sites.uci.edu/aealper...ft_aug2014.pdf

Instead of using leads and lags, could you control for the anticipatory effect by performing a DiD on a placebo date prior to the treatment date. For example, on the announcement date?

Many thanks!

[Oussama Ben Atta]

Dears,

Within the context of the placebo test by including lead variables, should the treatment variable t be kept in the specification or should it be deleted?

Thanks,