Matching on pre-trends for difference-in-differences regression with staggered treatment

[Katarina Sandberg]

Hi!
I have a panel data set of municipalities between 1998-2021, where I want to estimate the effect of an intervention that happened at different times in some of the municipalities (i.e. the treatment is staggered). Here I employ the approach by Callaway and Sant’Anna (2021) and the csdid package, which have worked great.

However, I have also been asked to employ the “regular” two way fixed effect-model, which does not work so well. I have run the TWFE-regression with leads and lags built on the notion that if the trend is parallel, the leads should not be statistically different from zero. However, one of them is. Furthermore, the overall model (i.e. running the TWFE-model without leads and lags) shows no significant signs of treatment.

I suspect that the lack of treatment effect could be because the municipalities that are selected for treatment significantly differ from the ones that are not selected for treatment. Hence, I would like to implement some kind of matching procedure, to ensure that the treated and control group are more similar.
However, I am not sure how to do this in a panel data setting.

My steps this far involve:
1. Run a logit regression where the outcome is ever being treated (1999-2021), and where the other variables are based on pre-treatment characteristics (i.e. the year 1998) that are believed to affect assignment into treatment.
2. Then propensity scores are assigned.
3. I then assign all of the observations within the same municipality to the same pscore.
4. Then I use psmatch2 to achieve weights, which I incorporate in the TWFE-regression.
5. I then incorporate leads and lags of treatment to see if the difference in pre-trends is significant between treated and untreated observations.

Code looks like this:
logit ever_treated matching variables if year==1998
predict pscore if year==1998
by municipalcode (year), sort: replace pscore = pscore[_n-1] if pscore >= .
psmatch2 ever_treat, pscore(pscore) outcome(depvar) neighbor(20)
xtreg depvar indepvar i.year [aweight=_weight], fe vce(cluster municipality)
xtreg depvar lead_5 lead_4 lead_3 lead_2 lead_1 lag0 lag1 lag2 lag3 lag4 lag5 i.year [aweight=_weight], fe vce(cluster municipality)

However, the results turn out almost identical to the unmatched setting.

My questions thus are:
1. How should one do an appropriate matching in the panel data setting with staggered treatment?
2. Is it working to use leads and lags with weights or should I show parallel trends in some other way with the matched sample? How?

[FernandoRios]

Using weights in Simple regression will depend if you have a single treatment cohort or multiple.
If you have a single treatment cohort, then doing what you did is fine.
However, if you have multiple treatment groups, the matching weights would balance to the full set of treated units, not specific treated units as CS does.
HTH
F

[Katarina Sandberg]

Thank you! So if I understand you correctly: Using weights is not an option for multiple cohorts?

[FernandoRios]

Its a bit more complicated than that
in CS, the weights are created separately to "reshape" the controls to look like treatments.
In the method you suggest you could reshpe the controls to look like a generic treatment group. But, if the different treatment groups look different, then the results you could get may be incorrect.
F

[Jared Greathouse]

The other solution to this is to just use a staggered synthetic control design, using the allsynth command.

[Jacopo Tramontano]

Hello! I have a similar problem.

My understanding is that, for each cohort of treated observations, there needs to be a full set of control observations at different years t, which then need to be matched according to whether they are before or after year of treatment g. Hence, matching has to occur for every cohort in which t<g.

For different treatment years g, this leads to a multiplication of sets of control observations: you need some cohorts of controls to match with treated cohorts at some year g1, and you need those same cohorts, earmarked and weighed differently, to match with OTHER treated cohorts at some other year g2.

Then, different TWFE DiDs are implemented on different g and aggregated, given different weights, in the dynamic event study version, as in Callaway and Sant'Anna.

My question is: how do I implement this in practice? Should I create different dummy variables for observations that are different controls at different times, and match on those?