Verifying Parallel Trends in DiD Analysis

[Sara Cucaro]

Good morning everyone,

I'm having some difficulties verifying the parallel trends assumption for a DiD strategy. I normally use (and have done so in the past) the usual graphical representation and visual inspection of trends, simply plotting them in the following way:

        bysort year: egen revenue_controls = mean(revenue) if treatment == 0
        bysort year: egen revenue_treatment = mean(revenue) if treatment == 1
        line revenue_controls revenue_treatment year

However, according to many scholars (including some of my professors) merely observing trends is not enough to verify this assumption. I know that, when using the xtdidregress function, the following commands can be used to verify the parallel trends assumption and perform the granger test:

        xtdidreg (revenue) (treatment), group(id) time(year)
        estat trendplot
        estat ptrends
        estat granger

However, I was wondering whether the same tests could be done without the xtdidreg command. Lastly, I saw that many papers directly plot the difference between trends. How can this be done through stata? Forgive me if this question turns out to be too basic, unfortunately I'm not an expert in econometrics and have many difficulties applying this methodology. Thank you!

[Maxence Morlet]

I recommend doing it "manually"; this forces you to get a better understanding of your data, and how estimations react to certain modifications.

Wooldridge (2021) outlines a great procedure in order to test for the parallel trends hypothesis whilst allowing for great flexibility: https://papers.ssrn.com/sol3/papers....act_id=3906345.

However, you will never be able to prove that the parallel trends hypothesis holds beyond reasonable doubt. There can always be questions.

I have talked to many journal referees about this, and they have unanimously told me that when they review a DiD paper, they will be looking for the graph and some sort of formal test, but they are especially after a solid argumentation that convinces them that nonrandom selection into treatment did not occur, or that the policy was plausibly exogenous.

[Jared Greathouse]

The reason folks like me prefer synthetic controls (or alternatively, synthetic difference-in-differences) is that the explicit weighting mechanism attempts to force the trends to be parallel. While this isn't the end of the story, weighting methods like SCM allow us to try and objectively select and weigh donors, thus making PTA more plausible

[Sara Cucaro]

ratio.jpg Thank you both, I'm now reading the procedure described in Wooldridge (2021) and hope I'll be able to understand it in detail. Anyways I'll definitely try to prove them theoretically.

In addition to my question above, to be more specific, I want to plot the difference between trends in the following way. This is an example from Bendavid et. al. (2012). Do I simply need to calculate the difference in means and plot it? Or should I use a specific function?

[Maxence Morlet]

I haven't read the paper but this seems to be an event study. In this case, what you might want to do is regress the outcome on controls, unit and time fixed effects (if you have panel data) and an interaction between treatment and each time dummy. Choose a year of reference for the graph (usually one period before treatment) and then plot the coefficients on the interaction terms. This seems to be what they have done in the first graph you show.

This procedure is a lot more thorough in Wooldridge (2021) and better explained

There is a community-contributed command,

xtevent

, that might help you with this and complement "manual" analysis.

And Jared is absolutely right, if the parallel trends assumption does not seem to hold, you may want to resort to matching, plot the new trends with weights, and then run DiD with weights generated from the matching procedure.

[Sara Cucaro]

Hi Maxence, thank you! That's not much of an event study, the authors are analyzing the impact of a policy measure on a subset of African countries through a DiD analysis, but thank you anyway for the community-contributed command - I didn't know it!

The graph I'm trying to replicate is the second one, plotting the means difference between trends with confidence intervals. Sorry I wasn't really clear on this!

[Jared Greathouse]

u "https://github.com/scunning1975/mixtape/blob/master/smoking.dta?raw=true", clear

xtset state year, y

g treated = cond(state==3 & year >= 1989,1,0) // Treated

xtdidregress (cigsale) (treated), group(state) time(year)

estat trendplots, omeans

This is how you'd do PTA analysis via xtdidregress. We see the averages of California and its 38 donor pool states. I could've made this much sexier and dolled it up, but this is the basic idea. You can edit this graph, and make it as nice as you'd like.

EDIT: You can't have confidence intervals without estimation. Let's revisit my synth example.

u "https://github.com/scunning1975/mixtape/blob/master/smoking.dta?raw=true", clear

xtset state year, y
* Synth weights from Abadie et al 2010

//ssc inst allsynth, replace

allsynth cigsale beer lnincome retprice age15to24 cigsale(1988) cigsale(1980) cigsale(1975), ///
trunit(3) trperiod(1989) nested xperiod(1970(1)1988) fig keep(cali, replace)
*/

gen wtpaper = 1 if state==3 // CA

replace wtpaper = .091 if state==4 // CO

replace wtpaper = .106 if state==5 // CT

replace wtpaper = .224 if state==19 // MT

replace wtpaper = .247 if state ==21 // NV

replace wtpaper = 0.333 if state ==34 // UT

label var wtpaper "synth wts from Abadie 2010"

g treated = cond(state==3,1,0) // Treated

g post = cond(year >= 1989,1,0) // Post


// Just like normal DD so far. Just reweighted.

 cls

reg cigsale i.treated##i.post [aw=wtpaper]

//-19.1604
u cali, clear
g diff = _Y_treated- _Y_synthetic
mean diff if _t >= 1989
//-19.28803

twoway (connected diff _time, mcol(red) lcolor(black) lwidth(medthick)), xline(1988)

I am too lazy to calculate CIs at the moment, just know that it's possible and that if the trends are this parallel in the pre-period, you don't gotta worry about the CIs pre intervention.

[Jared Greathouse]

To demonstrate what I mean, here is the plot that looks more or less what you'd want, after estimating your treatment effect. These are estimated by scul, my synthetic control command.

* Example generated by -dataex-. For more info, type help dataex
clear
input float(relative diff_ te_ub te_lb)
-18   .9862013  10.502757 -8.5303545
-17  -.4987221   9.017834 -10.015278
-16 -.58215463   8.934401  -10.09871
-15   .3793476   9.895904  -9.137208
-14 .066485435   9.583041   -9.45007
-13   .4329072   9.949463  -9.083649
-12  .21162495   9.728181  -9.304931
-11  -.1997787   9.316777  -9.716334
-10   .7157747   10.23233  -8.800781
 -9  .21055834   9.727114  -9.305997
 -8 -.25033668   9.266219  -9.766892
 -7  .22281595   9.739371  -9.293739
 -6  -.1818499   9.334705  -9.698405
 -5  -.6480923  8.8684635 -10.164648
 -4   .5497132   10.06627  -8.966843
 -3  -.4755211   9.041035  -9.992077
 -2  -.6018732   8.914682  -10.11843
 -1     -.3371   9.179456  -9.853656
  0       -4.9   4.616556 -14.416555
  1  -9.065703   .4508522 -18.582258
  2  -9.919065  -.4025099  -19.43562
  3 -12.772883  -3.256328  -22.28944
  4  -11.96314  -2.446584 -21.479694
  5 -16.440878  -6.924322 -25.957434
  6  -18.15645 -8.6398945 -27.673006
  7  -20.66679 -11.150236  -30.18335
  8  -20.75482 -11.238262 -30.271374
  9 -20.795294 -11.278738  -30.31185
 10 -23.710476  -14.19392  -33.22703
 11  -26.18433 -16.667776 -35.700886
 12  -27.09144 -17.574883 -36.607994
end

twoway (line diff relative, ///
lcolor(black) lwidth(thick)) ///
(rline te_lb te_ub rel, lcolor(red) lwidth(vthin) lpattern(shortdash)), ///
xli(-1, lpat(solid)) ///
yti("Pointwise Treatment Effect") ///
xti("t-`=ustrunescape("\u2113")' until Prop 99") ///
scheme(white_tableau) legend(order(1 "Treatment Effect" 2 "95% CIs") ///
pos(7) ring(0))