Testing for parallel trend

[Gaia Gaudenzi]

Hello statalists!


I have a very beginner question,
I am working on a panel dataset trying to estimate the effect of a policy that was implemented in some provinces in Argentina.
I would like to ask you how it is possible to test for parallel trends in a Diff in Diff case with lags and leads. I tried to look on the internet and I found the Autor paper, but I find it difficult to practically implement it with Stata.

The code I used for the Diff in Diff is

xtreg depvar DiD controls, fe cluster(clustervar) 

Thank you very much in advance!

[Clyde Schechter]

More information is needed in order to answer your question.

First of all, your -xtreg- command does not look like a correct DID analysis. You don't explain what the variable DiD is, but on the assumption that it is 1 for those observations where the province has already adopted the policy and 0 elsewhere, then your model is missing a time variable. You need to include some thing like i.post, or i.date, or i.year, or something like that. Without a time variable, your model is mis-specified.

Next, to get help with coding anything, you need to show example data. I can write code for imaginary data, but there is a good chance that the data I imagine will differ from your actual data enough that it won't work for you. Then we've both wasted our time. So when you post back, use the -dataex- command to show an example of your actual Stata data set. If you are running version 15.1 or a fully updated version 14.2, -dataex- is already part of your official Stata installation. If not, run -ssc install dataex- to get it. Either way, run -help dataex- to read the simple instructions for using it. -dataex- will save you time; it is easier and quicker than typing out tables. It includes complete information about aspects of the data that are often critical to answering your question but cannot be seen from tabular displays or screenshots. It also makes it possible for those who want to help you to create a faithful representation of your example to try out their code, which in turn makes it more likely that their answer will actually work in your data.

When asking for help with code, always show example data. When showing example data, always use -dataex-.

Next, it is somewhat unusual to "test" for parallel trends. The reason for this is that parallel trend is a null hypothesis: it is the hypothesis that in the pre-policy period, the trend in the treatment group equals that in the control group. It is more common to simply show a graph of the trends in both groups prior to policy implementation. If you really want to do a formal test, you can do that, but it requires first doing a power calculation, because failure to reject the null hypothesis of parallel trends could simply mean that your data are too noisy or too scanty to mount a rejection.

[Gaia Gaudenzi]

Dear Clyde,

thank you for your help, in effect my full code already accounted for a year interaction

xtreg renmat DiD i.year controls, fe cluster(Province) 

where DiD is a variable that takes the value of 1 if the province has reached more than 50% of students under the reform

the data sample is provided here

* Example generated by -dataex-. To install: ssc install dataex
clear
input long idwebest str21 Province double(year renmat renlen DiD)
132430 " CAPITAL FEDERAL     " 1997 46.819999972979225  56.78291642665863 0
207700 " CAPITAL FEDERAL     " 1997 60.176666005452475  53.48888905843099 0
200585 " CAPITAL FEDERAL     " 1997  69.94473728380706  75.10352998621323 0
139050 " CAPITAL FEDERAL     " 1997  68.02649955749511  74.41842109278629 0
177805 " CAPITAL FEDERAL     " 1997                  .          67.296875 0
134480 " CAPITAL FEDERAL     " 1997   67.4640906940807  75.83391239332116 0
126365 " CAPITAL FEDERAL     " 1997  57.89347826916239  77.67600040435791 0
160530 " CAPITAL FEDERAL     " 1997   69.1386363289573  77.90681856328791 0
159600 " CAPITAL FEDERAL     " 1997 55.264000193277994  63.61176434685202 0
138870 " CAPITAL FEDERAL     " 1997 58.171578758641296  70.54277780320909 0
192090 " CAPITAL FEDERAL     " 1997  39.76500002543131  36.79882330052993 0
163070 " CAPITAL FEDERAL     " 1997  41.28250026702881  44.33062517642975 0
177705 " CAPITAL FEDERAL     " 1997  66.98636263067073  77.07571520124164 0
111605 " CAPITAL FEDERAL     " 1997  49.85388843218485 55.539411544799805 0
224505 " CAPITAL FEDERAL     " 1997   82.8003570011684  85.24230722280649 0
155630 " CAPITAL FEDERAL     " 1997  75.68559982299804  75.31384622133695 0
238970 " CAPITAL FEDERAL     " 1997  52.86954578486356 59.468094961983816 0
235905 " CAPITAL FEDERAL     " 1997  67.36733322143554  61.01294158486759 0
238085 " CAPITAL FEDERAL     " 1997  47.66222222646078  58.43062508106232 0
213750 " CAPITAL FEDERAL     " 1997 63.158094951084685  67.12409175526012 0
103840 " CAPITAL FEDERAL     " 1997                  .  79.84444385104709 0
 97870 " CAPITAL FEDERAL     " 1997  66.66666679382324  60.33611096276177 0
231680 " CAPITAL FEDERAL     " 1997  59.00263133801912  71.44722146458096 0
207255 " CAPITAL FEDERAL     " 1997  83.61272742531516   87.6639115706734 0
199035 " CAPITAL FEDERAL     " 1997  72.73719940185546  76.32090915333141 0
128965 " CAPITAL FEDERAL     " 1997  52.24259263497812  64.77259296841092 0
235935 " CAPITAL FEDERAL     " 1997  67.15759994506836  67.63624986012776 0
157535 " CAPITAL FEDERAL     " 1997 60.929230616642876 59.034616030179535 0
132720 " CAPITAL FEDERAL     " 1997  72.91666634877522   83.8984603881836 0
186970 " CAPITAL FEDERAL     " 1997   81.9538094656808  72.22222174538507 0
112065 " CAPITAL FEDERAL     " 1997    64.603999710083   76.0099993254009 0
135175 " CAPITAL FEDERAL     " 1997  78.37521752067234  74.41857160840716 0
200380 " CAPITAL FEDERAL     " 1997   80.1166667231807  79.53433329264323 0
105705 " CAPITAL FEDERAL     " 1997  74.38666687011718   66.4626310248124 0
125925 " CAPITAL FEDERAL     " 1997  56.89758603326206   67.5227584510014 0
177200 " CAPITAL FEDERAL     " 1997 27.631666951709324  40.00050010681152 0
181335 " CAPITAL FEDERAL     " 1997  49.12222284740872  59.91705849591423 0
128860 " CAPITAL FEDERAL     " 1997  57.17727288332853  59.68904722304571 0
183800 " CAPITAL FEDERAL     " 1997  69.73642839704242  58.26944457160102 0
112950 " CAPITAL FEDERAL     " 1997   59.8685001373291  70.54238019670758 0
128735 " CAPITAL FEDERAL     " 1997  70.06437492370605  58.72125029563904 0
  7530 " CAPITAL FEDERAL     " 1997  37.01866683959961 47.358182733709164 0
141030 " CAPITAL FEDERAL     " 1997   54.4406253695488  72.52937507629395 0
103945 " CAPITAL FEDERAL     " 1997  55.81789458425421  52.84277788798014 0
186290 " CAPITAL FEDERAL     " 1999  73.23076893733098              80.84 0
163600 " CAPITAL FEDERAL     " 1999  62.61111089918349             66.625 0
164905 " CAPITAL FEDERAL     " 1999  45.34615340599647   65.1199998474121 0
146190 " CAPITAL FEDERAL     " 1999  76.23999984741211  85.47826070370881 0
207400 " CAPITAL FEDERAL     " 1999  83.45833333333333  81.58333333333333 0
110800 " CAPITAL FEDERAL     " 1999 60.439999694824216  68.16666603088379 0
162560 " CAPITAL FEDERAL     " 1999                 23  47.54999980926514 0
112915 " CAPITAL FEDERAL     " 1999   67.0476188659668  67.28571374075753 0
211080 " CAPITAL FEDERAL     " 1999  79.05555544959174  82.03030279910925 0
 15250 " CAPITAL FEDERAL     " 1999  48.60714258466448  53.14285673413958 0
175255 " CAPITAL FEDERAL     " 1999  70.91304331240447   64.7391301030698 0
104760 " CAPITAL FEDERAL     " 1999  62.93749976158142  66.11764683442958 0
207000 " CAPITAL FEDERAL     " 1999   48.7894730818899 61.736841904489616 0
 89395 " CAPITAL FEDERAL     " 1999  83.33333333333333               90.9 0
218445 " CAPITAL FEDERAL     " 1999                 89  86.72727272727273 0
132230 " CAPITAL FEDERAL     " 1999  74.35714272090367  75.67741910872921 0
105805 " CAPITAL FEDERAL     " 1999 58.263157292416224  65.47368400975277 0
116845 " CAPITAL FEDERAL     " 1999   71.5555551317003  67.88235271678252 0
229265 " CAPITAL FEDERAL     " 1999              71.88  84.74074074074075 0
238525 " CAPITAL FEDERAL     " 1999  70.86666653951009  67.96428557804653 0
192075 " CAPITAL FEDERAL     " 1999   51.5555551317003  63.21052611501593 0
195510 " CAPITAL FEDERAL     " 1999   75.3913043478261  84.56521722544794 0
168765 " CAPITAL FEDERAL     " 1999  74.21428516932896  86.17241366156216 0
202160 " CAPITAL FEDERAL     " 1999  83.93103448275862  79.57142802647182 0
105600 " CAPITAL FEDERAL     " 1999 45.555554919772675 47.705882128547216 0
140595 " CAPITAL FEDERAL     " 1999 53.791666666666664  64.15999954223633 0
189395 " CAPITAL FEDERAL     " 1999  83.49999978807237  80.36842105263158 0
218365 " CAPITAL FEDERAL     " 1999  62.87999954223633  72.15384542025052 0
163910 " CAPITAL FEDERAL     " 1999  72.05263157894737  73.72222222222223 0
128665 " CAPITAL FEDERAL     " 1999   56.5555551317003 59.666666666666664 0
224490 " CAPITAL FEDERAL     " 1999  66.41666634877522               61.6 0
167160 " CAPITAL FEDERAL     " 1999  54.76190476190476  58.29999942779541 0
217275 " CAPITAL FEDERAL     " 1999  60.49999918256487   71.3636360168457 0
198680 " CAPITAL FEDERAL     " 1999  69.99999960537615  80.16666641235352 0
206140 " CAPITAL FEDERAL     " 1999  48.24999976158142  68.43749976158142 0
  7630 " CAPITAL FEDERAL     " 1999   59.8235294117647  69.31578907213714 0
195685 " CAPITAL FEDERAL     " 1999  60.06666615804036   65.3529411764706 0
167215 " CAPITAL FEDERAL     " 1999  55.52173813529637  64.04999923706055 0
167105 " CAPITAL FEDERAL     " 1999  69.07142857142857  69.33333269755046 0
146345 " CAPITAL FEDERAL     " 1999  75.91666645473904  85.54054054054055 0
 94030 " BUENOS AIRES        " 1997   73.0463630213882  70.11969791759144 1
149645 " BUENOS AIRES        " 1997 37.170624911785126  41.99588214649874 1
179055 " BUENOS AIRES        " 1997  78.84239959716797  59.57115422762357 1
115060 " BUENOS AIRES        " 1997   68.8454835953251  76.81593632698059 1
147820 " BUENOS AIRES        " 1997  59.21055560641818 46.785882613238165 1
221670 " BUENOS AIRES        " 1997 40.350416680177055 58.140000301858656 1
167645 " BUENOS AIRES        " 1997 61.484091238542035  76.42772778597745 1
 95660 " BUENOS AIRES        " 1997  78.42149982452392  85.04952348981585 1
 93700 " BUENOS AIRES        " 1997  49.57879981994629  64.82666699091594 1
 86235 " BUENOS AIRES        " 1997  72.53312540054321  81.02359985351562 1
155555 " BUENOS AIRES        " 1997 42.983333587646484  60.35285686311268 1
149040 " BUENOS AIRES        " 1997  73.97888840569391  74.54833369784885 1
124970 " BUENOS AIRES        " 1997 30.030588178073657 30.879444228278267 1
 88490 " BUENOS AIRES        " 1997  50.29111099243164  68.92181847312234 1
206650 " BUENOS AIRES        " 1997 30.100000023841858  35.37368435608713 1
205865 " BUENOS AIRES        " 1997  58.69565225684124  69.87363589893688 1
end

Every line is a school and I want to evaluate the impact of a reform that occurred in 1996. There was a staggered implementation so BUENOS AIRES happened to be one of the first ones implementing and CAPITAL FEDERAL never implemented it.

Standard errors are clustered at province level

I already graphed the data including one period before the beginning of the reform, and, visually, they seem to be parallel. However I wanted to test this hypothesis formally, and I was wondering how to do it.

Thank you very much for your help

[Clyde Schechter]

So if you want to formally test this you can do it like this:

label define reformed    0    "CAPITAL FEDERAL"    1    "BUENOS AIRES"
gen byte reformed:reformed = trim(itrim(Province)) == "BUENOS AIRES"

xtreg renmat c.year##i.reformed  if year < 1996, fe

Notes:

1. If you have more than two provinces in the real data, then you have to define the variable reformed so that it is 1 for those that ultimately adopted the reform, and 0 for those that did not.

2. If renmat does not follow a more or less linear trend in the pre-1996 era, then you need to change c.year to i.year and then you have to add -testparm i.year#i.reformed- to the code to get the actual test.

This code is not properly tested, but I believe it is right. The data example you posted is not suitable for testing parallel trends because, a) it contains no observations before 1996, and b) you need to have data from more than just one year in the pre-reform era, and you need to have it for both the group that did and the group that did not adopt the reform.

That is the code to do the formal test. A "statistically significant" result rejecting the null hypothesis of no difference would say that the data appear to be inconsistent with the parallel trends assumption. Remember, however, that if you end up not rejecting that null hypothesis, you cannot claim to have affirmed the parallel trends assumption unless you have first done a power analysis showing that you have adequate power to achieve such a rejection.

Added: I have used -xtreg, fe- as the regression command. But I am not sure I understand the structure of your data. If you have repeated observations on schools, which are, in turn, nested in provinces, then you would need to use a three-level model, using -mixed- instead of -xtreg-. So you need to figure out what the nesting structure of your observations is and pick an appropriate command for the regression.

[TANVEER Khan]

I have a similar issue, I also want to test the parallel trend assumption using leads and lags. I am working on a panel data from 2000-2010 [0-10 here] and I want to find the causal impact of MFA(2005) on export performance. I run the following model,
xtreg lnxear time did, fe r where lnxear is export earnings and did an interaction term. Please help me how I can do it.

* Example generated by -dataex-. To install: ssc install dataex
clear
input int id byte year float(firevind lnxear) byte(time treatoldi dido)
 68 3 .46 1.7702384 0 0 0
116 2 .46  3.259842 0 0 0
 99 4 .98 1.8789656 0 0 0
 71 0   0  2.863684 0 0 0
 68 2 .46  1.757724 0 0 0
 81 3 .98         . 0 0 0
156 4 .95 2.2575393 0 0 0
 98 4 .98 2.5390434 0 0 0
100 4 .97   2.44825 0 0 0
161 1 .98         . 0 0 0
 80 0 .19  1.953684 0 0 0
145 2   0 2.8484735 0 0 0
 60 2  .9  2.834349 0 0 0
 42 1  .9         . 0 0 0
110 4   0  1.506701 0 0 0
 72 1 .46  1.723248 0 0 0
 28 1 .34         . 0 0 0
140 1 .46  3.165075 0 0 0
157 4   1  2.488218 0 0 0
 90 3 .98         . 0 0 0
  6 1 .98         . 0 0 0
167 2   0         . 0 0 0
 36 4   0  2.984693 0 0 0
 18 0   0  2.815094 0 0 0
 83 4 .46  3.041361 0 0 0
149 1 .97         . 0 0 0
 18 1   0  2.716725 0 0 0
  5 4   0 2.9711056 0 0 0
123 2 .46 3.2297995 0 0 0
124 1 .97  3.516292 0 0 0
134 0   0 2.0342274 0 0 0
 38 1  .9  2.888966 0 0 0
 10 4   0  1.490311 0 0 0
103 4   0  1.954003 0 0 0
147 1 .97  2.409351 0 0 0
 78 1   0 2.8767316 0 0 0
102 0   0  2.461132 0 0 0
120 4 .46   2.86483 0 0 0
 91 3 .46  2.740151 0 0 0
 82 2   0  3.180484 0 0 0
107 1   0  3.031368 0 0 0
101 0 .97 1.9454114 0 0 0
 41 1   0  1.807569 0 0 0
142 0 .46   2.41946 0 0 0
149 3 .97         . 0 0 0
 62 4 .34  2.758497 0 0 0
 51 4 .46 2.3673167 0 0 0
 99 1 .98 1.5518523 0 0 0
 99 3 .98  1.621089 0 0 0
 30 1   0         . 0 0 0
 68 1 .46  1.713703 0 0 0
 12 3   1  2.907424 0 0 0
 36 1   0 2.9062085 0 0 0
185 0   1         . 0 0 0
121 1  .9  2.997148 0 0 0
114 4 .46  2.859509 0 0 0
 87 2   0         . 0 0 0
 23 1 .46   2.57938 0 0 0
 14 2 .98 1.7721617 0 0 0
  3 0 .98  2.873453 0 0 0
  1 1   1  3.464713 0 0 0
 26 3   0         . 0 0 0
187 1  .4         . 0 0 0
115 3 .46 3.5667944 0 0 0
192 1 .46         . 0 0 0
 26 0   0         . 0 0 0
 75 2   1 1.5459526 0 0 0
112 1 .46  3.280337 0 0 0
 62 1 .34   2.66255 0 0 0
 39 1 .98 2.0091832 0 0 0
158 3 .34         . 0 0 0
 64 1 .46         . 0 0 0
 88 2 .46         . 0 0 0
152 1 .46 1.5675745 0 0 0
131 0   1 2.9316664 0 0 0
155 1 .46 2.1249473 0 0 0
113 1 .46  3.345758 0 0 0
 78 0   0  2.892543 0 0 0
116 1 .46  3.176337 0 0 0
132 4  .9  2.494881 0 0 0
 29 1 .46         . 0 0 0
 92 0 .46         . 0 0 0
 40 1  .4 2.1206496 0 0 0
 14 3 .98 2.1538148 0 0 0
106 4 .46         . 0 0 0
184 1 .97  2.562722 0 0 0
 18 2   0  2.507439 0 0 0
107 0   0 3.0423865 0 0 0
 94 4 .46         . 0 0 0
118 4 .97         . 0 0 0
187 0  .4         . 0 0 0
 20 2   1 2.2031307 0 0 0
119 1   0  3.425684 0 0 0
113 2 .46  3.549464 0 0 0
 95 1   1         . 0 0 0
184 2 .97  2.845985 0 0 0
104 3   0 2.3705575 0 0 0
 77 1 .98 4.1635313 0 0 0
 27 2 .46 2.2013726 0 0 0
187 4  .4  1.943835 0 0 0
end

------------------ copy up to and including the previous line ------------------

Listed 100 out of 2112 observations
Use the count() option to list more

.