Dear All,
I am trying to evaluate the impact of a treatment using difference-in-difference and event study design. My sense is that diff-in-diff captures average treatment effect in treated and event study is the LATE. With that in mind my data is as follows:
I am unable to include all controls due to dataex linesize limits. But I hope you get the idea. Then I run the diff-in-diff estimation as follows:
The variable post captures the diff-in-diff estimate (treated *period after treatment) and happens to be statistically significant in this case.
I estimate the event study as follows:
The event* coefficient capture the event-time effects with event_qtr5 omitted as the reference period. Unlike the DD, here I find not statistically significant effect of treatment at any of the post treatment periods (event_qtr6, event_qtr7, event_qtr8 and event_qtr9). My sense is that the two models should be complimentary and capture similar results. Here, that is not the case. I would be grateful for any insights into why this could be the case.
Sincerely,
Sumedha.
I am trying to evaluate the impact of a treatment using difference-in-difference and event study design. My sense is that diff-in-diff captures average treatment effect in treated and event study is the LATE. With that in mind my data is as follows:
Code:
* Example generated by -dataex-. To install: ssc install dataex clear input float(total_amnt_reimbursed2 post) byte treatMA float(Medicaidbeneficiaries2 qavg_pct_lf_unemp pct_lhs pct_hs perc_black perc_nonwhite pctmale pctover65) double pop_total float(polydrug_type qtr) byte stateFIPS float qtrsq 244346.45 0 1 18705.248 7.733333 7.7 27.7 4.545096 28.878407 .52014977 .08111796 722713 1 1 2 1 118670.2 0 1 18705.248 7.733333 7.7 27.7 4.545096 28.878407 .52014977 .08111796 722713 2 1 2 1 133113.53 0 1 18705.248 7.733333 7.7 27.7 4.545096 28.878407 .52014977 .08111796 722713 3 1 2 1 127715.05 0 1 18705.248 7.733333 7.7 27.7 4.545096 28.878407 .52014977 .08111796 722713 4 1 2 1 17496.982 0 1 18705.248 7.733333 7.7 27.7 4.545096 28.878407 .52014977 .08111796 722713 5 1 2 1 315921.9 0 1 18705.248 7.566667 7.7 27.7 4.545096 28.878407 .52014977 .08111796 722713 1 2 2 4 189714.88 0 1 18705.248 7.566667 7.7 27.7 4.545096 28.878407 .52014977 .08111796 722713 2 2 2 4 126031.88 0 1 18705.248 7.566667 7.7 27.7 4.545096 28.878407 .52014977 .08111796 722713 3 2 2 4 127737.7 0 1 18705.248 7.566667 7.7 27.7 4.545096 28.878407 .52014977 .08111796 722713 4 2 2 4 14322.278 0 1 18705.248 7.566667 7.7 27.7 4.545096 28.878407 .52014977 .08111796 722713 5 2 2 4 293796.7 0 1 18705.248 7.5 7.7 27.7 4.545096 28.878407 .52014977 .08111796 722713 1 3 2 9 124752.18 0 1 18705.248 7.5 7.7 27.7 4.545096 28.878407 .52014977 .08111796 722713 2 3 2 9 116695.52 0 1 18705.248 7.5 7.7 27.7 4.545096 28.878407 .52014977 .08111796 722713 3 3 2 9 122939.16 0 1 18705.248 7.5 7.7 27.7 4.545096 28.878407 .52014977 .08111796 722713 4 3 2 9 10669.127 0 1 18705.248 7.5 7.7 27.7 4.545096 28.878407 .52014977 .08111796 722713 5 3 2 9 276289.63 0 1 18705.248 7.466667 7.7 27.7 4.545096 28.878407 .52014977 .08111796 722713 1 4 2 16 202972.84 0 1 18705.248 7.466667 7.7 27.7 4.545096 28.878407 .52014977 .08111796 722713 2 4 2 16 118347.64 0 1 18705.248 7.466667 7.7 27.7 4.545096 28.878407 .52014977 .08111796 722713 3 4 2 16 101024.47 0 1 18705.248 7.466667 7.7 27.7 4.545096 28.878407 .52014977 .08111796 722713 4 4 2 16 8189.157 0 1 18705.248 7.466667 7.7 27.7 4.545096 28.878407 .52014977 .08111796 722713 5 4 2 16 171985.44 0 1 14950.732 7.333333 7.7 27.7 4.6983337 29.19467 .5209735 .0854725 731089 1 5 2 25 62334.35 0 1 14950.732 7.333333 7.7 27.7 4.6983337 29.19467 .5209735 .0854725 731089 2 5 2 25 135478.81 0 1 14950.732 7.333333 7.7 27.7 4.6983337 29.19467 .5209735 .0854725 731089 3 5 2 25 111281.08 0 1 14950.732 7.333333 7.7 27.7 4.6983337 29.19467 .5209735 .0854725 731089 4 5 2 25 9459.741 0 1 14950.732 7.333333 7.7 27.7 4.6983337 29.19467 .5209735 .0854725 731089 5 5 2 25 131014.23 0 1 14950.732 7.166667 7.7 27.7 4.6983337 29.19467 .5209735 .0854725 731089 1 6 2 36 47940.76 0 1 14950.732 7.166667 7.7 27.7 4.6983337 29.19467 .5209735 .0854725 731089 2 6 2 36 132750.38 0 1 14950.732 7.166667 7.7 27.7 4.6983337 29.19467 .5209735 .0854725 731089 3 6 2 36 95422.48 0 1 14950.732 7.166667 7.7 27.7 4.6983337 29.19467 .5209735 .0854725 731089 4 6 2 36 7618.546 0 1 14950.732 7.166667 7.7 27.7 4.6983337 29.19467 .5209735 .0854725 731089 5 6 2 36 145094.31 0 1 14950.732 7.033333 7.7 27.7 4.6983337 29.19467 .5209735 .0854725 731089 1 7 2 49 61439.27 0 1 14950.732 7.033333 7.7 27.7 4.6983337 29.19467 .5209735 .0854725 731089 2 7 2 49 116494.5 0 1 14950.732 7.033333 7.7 27.7 4.6983337 29.19467 .5209735 .0854725 731089 3 7 2 49 99968.64 0 1 14950.732 7.033333 7.7 27.7 4.6983337 29.19467 .5209735 .0854725 731089 4 7 2 49 7904.148 0 1 14950.732 7.033333 7.7 27.7 4.6983337 29.19467 .5209735 .0854725 731089 5 7 2 49 178488.3 0 1 14950.732 7 7.7 27.7 4.6983337 29.19467 .5209735 .0854725 731089 1 8 2 64 95142.49 0 1 14950.732 7 7.7 27.7 4.6983337 29.19467 .5209735 .0854725 731089 2 8 2 64 121128.34 0 1 14950.732 7 7.7 27.7 4.6983337 29.19467 .5209735 .0854725 731089 3 8 2 64 83648.16 0 1 14950.732 7 7.7 27.7 4.6983337 29.19467 .5209735 .0854725 731089 4 8 2 64 9174.959 0 1 14950.732 7 7.7 27.7 4.6983337 29.19467 .5209735 .0854725 731089 5 8 2 64 423187.2 0 1 14870.19 7 7.7 27.7 4.786403 29.4967 .5228226 .08950587 736879 1 9 2 81 248851.05 0 1 14870.19 7 7.7 27.7 4.786403 29.4967 .5228226 .08950587 736879 2 9 2 81 129872.72 0 1 14870.19 7 7.7 27.7 4.786403 29.4967 .5228226 .08950587 736879 3 9 2 81 92456.37 0 1 14870.19 7 7.7 27.7 4.786403 29.4967 .5228226 .08950587 736879 4 9 2 81 9958.515 0 1 14870.19 7 7.7 27.7 4.786403 29.4967 .5228226 .08950587 736879 5 9 2 81 359735.2 0 1 14870.19 7 7.7 27.7 4.786403 29.4967 .5228226 .08950587 736879 1 10 2 100 228691.1 0 1 14870.19 7 7.7 27.7 4.786403 29.4967 .5228226 .08950587 736879 2 10 2 100 119236.2 0 1 14870.19 7 7.7 27.7 4.786403 29.4967 .5228226 .08950587 736879 3 10 2 100 92072.89 0 1 14870.19 7 7.7 27.7 4.786403 29.4967 .5228226 .08950587 736879 4 10 2 100 9407.232 0 1 14870.19 7 7.7 27.7 4.786403 29.4967 .5228226 .08950587 736879 5 10 2 100 327640.7 0 1 14870.19 7 7.7 27.7 4.786403 29.4967 .5228226 .08950587 736879 1 11 2 121 225564.95 0 1 14870.19 7 7.7 27.7 4.786403 29.4967 .5228226 .08950587 736879 2 11 2 121 108184.56 0 1 14870.19 7 7.7 27.7 4.786403 29.4967 .5228226 .08950587 736879 3 11 2 121 86647.55 0 1 14870.19 7 7.7 27.7 4.786403 29.4967 .5228226 .08950587 736879 4 11 2 121 6760.568 0 1 14870.19 7 7.7 27.7 4.786403 29.4967 .5228226 .08950587 736879 5 11 2 121 83992.07 0 1 14870.19 7 7.7 27.7 4.786403 29.4967 .5228226 .08950587 736879 1 12 2 144 12392.716 0 1 14870.19 7 7.7 27.7 4.786403 29.4967 .5228226 .08950587 736879 2 12 2 144 126082.85 0 1 14870.19 7 7.7 27.7 4.786403 29.4967 .5228226 .08950587 736879 3 12 2 144 80156.08 0 1 14870.19 7 7.7 27.7 4.786403 29.4967 .5228226 .08950587 736879 4 12 2 144 9189.192 0 1 14870.19 7 7.7 27.7 4.786403 29.4967 .5228226 .08950587 736879 5 12 2 144 86457.52 0 1 15967.253 7 7.7 27.7 4.795406 29.827 .5233248 .09415574 736705 1 13 2 169 12616.342 0 1 15967.253 7 7.7 27.7 4.795406 29.827 .5233248 .09415574 736705 2 13 2 169 121108.2 0 1 15967.253 7 7.7 27.7 4.795406 29.827 .5233248 .09415574 736705 3 13 2 169 69447.66 0 1 15967.253 7 7.7 27.7 4.795406 29.827 .5233248 .09415574 736705 4 13 2 169 8063.893 0 1 15967.253 7 7.7 27.7 4.795406 29.827 .5233248 .09415574 736705 5 13 2 169 92839.48 0 1 15967.253 7 7.7 27.7 4.795406 29.827 .5233248 .09415574 736705 1 14 2 196 15066.254 0 1 15967.253 7 7.7 27.7 4.795406 29.827 .5233248 .09415574 736705 2 14 2 196 118625.94 0 1 15967.253 7 7.7 27.7 4.795406 29.827 .5233248 .09415574 736705 3 14 2 196 66836.164 0 1 15967.253 7 7.7 27.7 4.795406 29.827 .5233248 .09415574 736705 4 14 2 196 9317.746 0 1 15967.253 7 7.7 27.7 4.795406 29.827 .5233248 .09415574 736705 5 14 2 196 89983.84 0 1 15967.253 6.866667 7.7 27.7 4.795406 29.827 .5233248 .09415574 736705 1 15 2 225 17058.08 0 1 15967.253 6.866667 7.7 27.7 4.795406 29.827 .5233248 .09415574 736705 2 15 2 225 97259.93 0 1 15967.253 6.866667 7.7 27.7 4.795406 29.827 .5233248 .09415574 736705 3 15 2 225 60028.43 0 1 15967.253 6.866667 7.7 27.7 4.795406 29.827 .5233248 .09415574 736705 4 15 2 225 9043.694 0 1 15967.253 6.866667 7.7 27.7 4.795406 29.827 .5233248 .09415574 736705 5 15 2 225 94355.04 0 1 15967.253 6.6 7.7 27.7 4.795406 29.827 .5233248 .09415574 736705 1 16 2 256 15371.92 0 1 15967.253 6.6 7.7 27.7 4.795406 29.827 .5233248 .09415574 736705 2 16 2 256 105806.2 0 1 15967.253 6.6 7.7 27.7 4.795406 29.827 .5233248 .09415574 736705 3 16 2 256 44462.72 0 1 15967.253 6.6 7.7 27.7 4.795406 29.827 .5233248 .09415574 736705 4 16 2 256 9933.933 0 1 15967.253 6.6 7.7 27.7 4.795406 29.827 .5233248 .09415574 736705 5 16 2 256 94429.52 0 1 17628.842 6.5 7.7 27.7 4.818702 30.10442 .5234001 .09878014 737709 1 17 2 289 18495.213 0 1 17628.842 6.5 7.7 27.7 4.818702 30.10442 .5234001 .09878014 737709 2 17 2 289 107516.88 0 1 17628.842 6.5 7.7 27.7 4.818702 30.10442 .5234001 .09878014 737709 3 17 2 289 41441.23 0 1 17628.842 6.5 7.7 27.7 4.818702 30.10442 .5234001 .09878014 737709 4 17 2 289 10527.1 0 1 17628.842 6.5 7.7 27.7 4.818702 30.10442 .5234001 .09878014 737709 5 17 2 289 95679.63 0 1 17628.842 6.5 7.7 27.7 4.818702 30.10442 .5234001 .09878014 737709 1 18 2 324 21429.29 0 1 17628.842 6.5 7.7 27.7 4.818702 30.10442 .5234001 .09878014 737709 2 18 2 324 117177.19 0 1 17628.842 6.5 7.7 27.7 4.818702 30.10442 .5234001 .09878014 737709 3 18 2 324 48050.66 0 1 17628.842 6.5 7.7 27.7 4.818702 30.10442 .5234001 .09878014 737709 4 18 2 324 10628.618 0 1 17628.842 6.5 7.7 27.7 4.818702 30.10442 .5234001 .09878014 737709 5 18 2 324 106074.66 0 1 17628.842 6.5 7.7 27.7 4.818702 30.10442 .5234001 .09878014 737709 1 19 2 361 29173.746 0 1 17628.842 6.5 7.7 27.7 4.818702 30.10442 .5234001 .09878014 737709 2 19 2 361 107673.52 0 1 17628.842 6.5 7.7 27.7 4.818702 30.10442 .5234001 .09878014 737709 3 19 2 361 45805.95 0 1 17628.842 6.5 7.7 27.7 4.818702 30.10442 .5234001 .09878014 737709 4 19 2 361 11432.882 0 1 17628.842 6.5 7.7 27.7 4.818702 30.10442 .5234001 .09878014 737709 5 19 2 361 94058.95 0 1 17628.842 6.633333 7.7 27.7 4.818702 30.10442 .5234001 .09878014 737709 1 20 2 400 26940.78 0 1 17628.842 6.633333 7.7 27.7 4.818702 30.10442 .5234001 .09878014 737709 2 20 2 400 103659.59 0 1 17628.842 6.633333 7.7 27.7 4.818702 30.10442 .5234001 .09878014 737709 3 20 2 400 43268.44 0 1 17628.842 6.633333 7.7 27.7 4.818702 30.10442 .5234001 .09878014 737709 4 20 2 400 10477.578 0 1 17628.842 6.633333 7.7 27.7 4.818702 30.10442 .5234001 .09878014 737709 5 20 2 400 end
Code:
eststo: reghdfe total_amnt_reimbursed2 post treatMA /*medianhouseholdincome*/ Medicai
> dbeneficiaries2 qavg_pct_lf_unemp pct_lhs pct_hs perc_black perc_nonwhite ///
> pctmale pctover65 state_share_rural_2010 md_100000 pa_100000 rn_100000 [weight=pop_tota
> l] if polydrug_type==2 , ///
> absorb(i.qtr i.stateFIPS i.stateFIPS#(c.qtr c.qtrsq)) vce(cluster stateFIPS)
(analytic weights assumed)
weight pop_total can only contain strictly positive reals, but 27 missing values were fou
> nd (will be dropped)
(converged in 12 iterations)
note: treatMA omitted because of collinearity
note: pct_lhs omitted because of collinearity
note: pct_hs omitted because of collinearity
note: state_share_rural_2010 omitted because of collinearity
HDFE Linear regression Number of obs = 1,376
Absorbing 3 HDFE groups F( 10, 50) = 58.88
Statistics robust to heteroskedasticity Prob > F = 0.0000
R-squared = 1.0000
Adj R-squared = 1.0000
Within R-sq. = 0.0385
Number of clusters (stateFIPS) = 51 Root MSE = 5936.9416
(Std. Err. adjusted for 51 clusters in stateFIPS)
----------------------------------------------------------------------------------------
| Robust
total_amnt_reimbursed2 | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-----------------------+----------------------------------------------------------------
post | 2008.464 985.0199 2.04 0.047 29.99348 3986.935
treatMA | 0 (omitted)
Medicaidbeneficiaries2 | -.2140429 .1856947 -1.15 0.255 -.5870217 .1589358
qavg_pct_lf_unemp | 827.0914 1090.566 0.76 0.452 -1363.376 3017.558
pct_lhs | 0 (omitted)
pct_hs | 0 (omitted)
perc_black | -448.3627 318.6635 -1.41 0.166 -1088.417 191.6919
perc_nonwhite | 627.2181 329.348 1.90 0.063 -34.29694 1288.733
pctmale | 504113.6 394769.6 1.28 0.208 -288804.4 1297032
pctover65 | 72387.47 64615.99 1.12 0.268 -57397.57 202172.5
state_share_rural_2010 | 0 (omitted)
md_100000 | 633.7607 720.4047 0.88 0.383 -813.2147 2080.736
pa_100000 | -1766.102 5410.897 -0.33 0.745 -12634.21 9102.005
rn_100000 | -70.15798 286.8057 -0.24 0.808 -646.2243 505.9083
----------------------------------------------------------------------------------------
Absorbed degrees of freedom:
-------------------------------------------------------------------------+
Absorbed FE | Num. Coefs. = Categories - Redundant |
-----------------------+-------------------------------------------------|
qtr | 27 27 0 |
stateFIPS | 0 51 51 * |
stateFIPS#c.qtr | 51 51 0 ? |
stateFIPS#c.qtrsq | 51 51 0 ? |
-------------------------------------------------------------------------+
? = number of redundant parameters may be higher
* = fixed effect nested within cluster; treated as redundant for DoF computation
(est3 stored)
I estimate the event study as follows:
Code:
eststo: reghdfe number_rx2 event_qtr1 event_qtr2 event_qtr3 event_qtr4 event_qtr6 event
> _qtr7 event_qtr8 event_qtr9 ///
> treatMA /*medianhouseholdincome*/ Medicaidbeneficiaries2 qavg_pct_lf_unemp pct_lhs
> pct_hs perc_black perc_nonwhite ///
> pctmale pctover65 state_share_rural_2010 md_100000 pa_100000 rn_100000 [weight=pop_tota
> l] if polydrug_type==2, ///
> absorb(i.qtr i.stateFIPS i.stateFIPS#(c.qtr c.qtrsq)) vce(cluster stateFIPS)
(analytic weights assumed)
weight pop_total can only contain strictly positive reals, but 27 missing values were fou
> nd (will be dropped)
(converged in 12 iterations)
note: treatMA omitted because of collinearity
note: pct_lhs omitted because of collinearity
note: pct_hs omitted because of collinearity
note: state_share_rural_2010 omitted because of collinearity
HDFE Linear regression Number of obs = 1,376
Absorbing 3 HDFE groups F( 17, 50) = 20.92
Statistics robust to heteroskedasticity Prob > F = 0.0000
R-squared = 1.0000
Adj R-squared = 1.0000
Within R-sq. = 0.0482
Number of clusters (stateFIPS) = 51 Root MSE = 214.4841
(Std. Err. adjusted for 51 clusters in stateFIPS)
----------------------------------------------------------------------------------------
| Robust
number_rx2 | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-----------------------+----------------------------------------------------------------
event_qtr1 | -10.41363 115.0324 -0.09 0.928 -241.463 220.6358
event_qtr2 | -35.03107 91.41801 -0.38 0.703 -218.6495 148.5874
event_qtr3 | 45.40047 82.30064 0.55 0.584 -119.9052 210.7062
event_qtr4 | 16.04128 56.39401 0.28 0.777 -97.22942 129.312
event_qtr6 | -10.00112 48.81185 -0.20 0.838 -108.0426 88.04037
event_qtr7 | -73.1491 97.15901 -0.75 0.455 -268.2987 122.0005
event_qtr8 | 20.65557 172.1345 0.12 0.905 -325.0868 366.3979
event_qtr9 | 49.11925 221.9319 0.22 0.826 -396.644 494.8825
treatMA | 0 (omitted)
Medicaidbeneficiaries2 | .0049802 .0109135 0.46 0.650 -.0169403 .0269007
qavg_pct_lf_unemp | -70.00779 51.01565 -1.37 0.176 -172.4757 32.46016
pct_lhs | 0 (omitted)
pct_hs | 0 (omitted)
perc_black | -12.88903 9.302762 -1.39 0.172 -31.57417 5.796122
perc_nonwhite | 3.068003 5.603374 0.55 0.586 -8.186706 14.32271
pctmale | 3619.773 13399.86 0.27 0.788 -23294.64 30534.18
pctover65 | 2452.461 1987.009 1.23 0.223 -1538.565 6443.486
state_share_rural_2010 | 0 (omitted)
md_100000 | 10.55331 22.87794 0.46 0.647 -35.39838 56.505
pa_100000 | 69.13893 191.876 0.36 0.720 -316.2554 454.5332
rn_100000 | -2.740151 9.421927 -0.29 0.772 -21.66465 16.18435
----------------------------------------------------------------------------------------
Absorbed degrees of freedom:
-------------------------------------------------------------------------+
Absorbed FE | Num. Coefs. = Categories - Redundant |
-----------------------+-------------------------------------------------|
qtr | 27 27 0 |
stateFIPS | 0 51 51 * |
stateFIPS#c.qtr | 51 51 0 ? |
stateFIPS#c.qtrsq | 51 51 0 ? |
-------------------------------------------------------------------------+
? = number of redundant parameters may be higher
* = fixed effect nested within cluster; treated as redundant for DoF computation
Sincerely,
Sumedha.
Comment