Using Matching for Difference in Difference Regression

[Thomas Wake]

Hi,

Please could someone advise me on how to incorporate matched variables into xtreg and xtdidreg analysis? I have had to use matching, as my pre-treatment trend is not parallel, and my results thus far without matching have been insignificant (p value of 0.105 for xtdidreg). I am hoping that by using matching, I will be able to achieve significance.

I have just matched my variables using:
"psmatch2 (Interaction dummy) (Area code) (Covariates) out (Dependent variable) common"
However, I am having trouble interpreting these results - how do I include this now into my regressions?

If someone could please tell me the relevant alterations I need to make that would be hugely appreciated!

[Jared Greathouse]

I don't like the framing of this very much. You shouldn't care about statistical significance at all. You should use matching for the first reason you described, namely to make PTA more plausible, not because your t-stat was less than 1.96. More broadly, we shouldn't use fancy analyses to achieve "significant" results, we should use them to correct design problems in our data. Otherwise, this is just blatantly admitting to p-hacking.

Anyways, please show me your current data using dataex and I'll look at what you've got. Luckily the solution is simple.

[Thomas Wake]

Hi Jared, thank you for your response.

Code used to generate: dataex Year LSOA logcrime logemploy loghealth loghousing logenviron logeducation Defunded Y2018 Defunded2018

* Example generated by -dataex-. For more info, type help dataex
clear
input int Year long LSOA float(logcrime logemploy loghealth loghousing logenviron logeducation) byte(Defunded Y2018 Defunded2018)

2010 1000019   2.1973357 -2.1202636   -.5915127  3.245968 3.5058186   3.419787 0 0 0
2015 1000019    2.302685 -1.6820086   -1.937942  3.437047  3.020961  3.2949846 0 0 0
2019 1000019   2.3025851 -1.9805017   -.7402388 3.7196026 3.2893715   3.262357 0 1 0
2010 1000030    3.496538   -1.89712   -.5987332  3.666462 4.1158667  3.5705876 0 0 0
2015 1000030     3.04457 -1.7092582   -.8580218 3.7568884   3.08571  3.6532004 0 0 0
2019 1000030    3.295837  -1.820159  -4.1351666 3.7975545  3.044189   3.657208 0 1 0
2010 1000051     3.33224 -1.5141277  -.04472836  3.473627 3.9952366   3.563232 0 0 0
2015 1000051    1.946053 -1.6296406   -.6217572  3.874529 3.3255396   3.390406 0 0 0
2019 1000051   1.3862944 -2.0402207   -.9390477 3.8285544  3.314949  3.3392155 0 1 0
2010 1000093   2.9957824  -1.609438 -.007344606   3.38233  3.720582  3.5884836 0 0 0
2015 1000093    2.833272 -1.4961092  -.25360277  3.674882 3.2388744   3.432599 0 0 0
2019 1000093   3.2580965 -1.7778566  -.17197527 3.9051394  3.774369   3.330596 0 1 0
2010 1000181   2.0795665  -2.813411           .  3.022832  3.783135 -.01958769 0 0 0
2015 1000181    1.946053  -3.101093           .  3.045522   3.47822 -1.0106014 0 0 0
2019 1000181   2.0794415 -3.2441936           .  3.035866  3.390271  -1.171183 0 1 0
2010 1000259    2.397986  -2.813411           .  3.247443 2.5852005   .6677217 0 0 0
2015 1000259    2.302685  -2.688248           .  3.412236 3.0348086  1.3972343 0 0 0
2019 1000259    .6931472 -2.7181005           .   3.40943  2.955587  1.3630258 0 1 0
2010 1000328   1.0989456  -2.995732           .  2.608232  2.604541  2.6191075 1 0 0
2015 1000328    2.397986  -2.813411           .  2.904603 2.1170986    2.82684 1 0 0
2019 1000328    2.833213  -3.015935           . 3.0136704 2.6092606  2.8455486 1 1 1
2010 1000386   2.1973357 -1.7719568  -.13005367  3.194772  3.802041  3.6577716 1 0 0
2015 1000386   2.1973357  -1.639897   -.4215945  3.403694  3.125883   3.685825 1 0 0
2019 1000386    2.890372 -1.6347557  -.10869942  3.605253 3.1062446  3.7067194 1 1 1
2010 1000401    2.639129  -2.207275  -1.6966878  2.952555  3.259934    3.56215 0 0 0
2015 1000401    1.791926 -1.9449106   -1.324259  3.280836   3.01067   3.323704 0 0 0
2019 1000401    2.564949 -2.3330443           .  3.240089   3.23589    3.25948 0 1 0
2010 1000432     2.48499 -2.3025851           .   2.77004  2.857501  3.7460005 1 0 0
2015 1000432    2.397986 -1.8773173           .  3.341801 2.4803965   3.568828 1 0 0
2019 1000432   3.3322046  -2.198225           . 3.3792584  2.836795  3.3195534 1 1 1
2010 1000460 .0009995003  -3.506558           .  3.014804  2.426584  1.5087794 0 0 0
2015 1000460     .693647  -3.381395           .  2.847406  1.516445  2.0636933 0 0 0
2019 1000460           0  -3.473768           . 3.2076936  2.083309  2.0882773 0 1 0
2010 1000483   2.9957824 -1.8325815   -.4675037  3.736555  3.881937   2.905882 1 0 0
2015 1000483     2.48499 -1.7897614  -1.2765435  3.707505   3.46411   3.041518 1 0 0
2019 1000483    3.178054  -1.937942           .  3.828446  3.217795   3.052302 1 1 1
2010 1000552     2.48499 -1.5606477  -.19332665    3.7224 4.1885056   2.938907 1 0 0
2015 1000552   2.9444916 -1.3056364    -.614336 3.7916164  3.922132  3.2569804 1 0 0
2019 1000552    3.178054 -1.6660082   -4.342806  3.745165 3.8357465   3.188499 1 1 1
2010 1000630   2.9444916 -1.8325815   -.8903957 3.7389994  3.667776  2.4547894 1 0 0
2015 1000630    2.833272 -1.9519283           . 3.8343856  3.146692  2.7752104 1 0 0
2019 1000630    3.583519  -2.225624           .  3.939813    3.1062   2.844153 1 1 1
2010 1000636   3.5835466 -2.4079456           .  3.621258  3.253313   2.684745 1 0 0
2015 1000636   2.1973357 -2.4079456           . 3.5182986  3.440322   2.788339 1 0 0
2019 1000636    2.944439  -2.847312           .  3.653874 3.5704055  2.9325795 1 1 1
2010 1000643   3.4340196  -1.469676   .09680906  3.720435  3.720909  2.7000046 0 0 0
2015 1000643   2.9957824 -1.3205066    .2350721   3.71557  3.600048  3.1875916 0 0 0
2019 1000643    3.583519  -1.469676  -.16723593  3.861971 3.4727426  3.0207176 0 1 0
2010 1000664    1.609638   -2.65926           .  3.503659 1.9960095  2.6748056 1 0 0
2015 1000664    2.397986 -2.2926347           .  3.622713 2.2168093    2.79783 1 0 0
2019 1000664    2.484907  -2.617296           .  3.739668  2.491634    2.75887 1 1 1
2010 1000677    3.218916  -3.218876           .  3.487832 3.5798976  1.5263253 1 0 0
2015 1000677    2.833272   -2.97593           . 3.3857114  3.361729   1.399704 1 0 0
2019 1000677    2.995732  -3.411248           . 3.4682016  3.443682  1.2781522 1 1 1
2010 1000721    2.772651   -1.89712    -.430555 3.1713636   3.03152  3.8144796 1 0 0
2015 1000721    2.639129  -1.461018   -.2823629  3.588617 2.5740616   3.545529 1 0 0
2019 1000721    3.367296 -1.6873995  -.57447565 3.4531255 2.8383765    3.71839 1 1 1
2010 1000725    1.946053 -2.3025851           . 3.0567706    3.0288   3.215815 1 0 0
2015 1000725    2.890427 -2.0174062           .  3.453379 3.1733344   3.209068 1 0 0
2019 1000725     2.70805 -2.1628232   -3.101093  3.428326   3.27918   3.149054 1 1 1
2010 1000735   2.9444916 -2.0402207  -.43735865  3.069533  3.045417   3.762105 0 0 0
2015 1000735    1.791926 -1.7259717    -1.89712 3.4369826  2.520756  3.8016276 0 0 0
2019 1000735    2.890372 -2.0714734   -6.214608  3.444065  3.157468    3.67663 0 1 0
2010 1000750    2.397986 -1.8325815   -.7946257 3.3559694 1.0582086   3.210722 0 0 0
2015 1000750    2.302685  -1.639897  -1.2039728 3.7184625   1.59615  3.4300115 0 0 0
2019 1000750    2.890372 -1.7897614           .  3.818218  2.123578   3.375469 0 1 0
2010 1000785    2.890427 -1.8325815    -.633911  2.942722  2.860839   3.766999 0 0 0
2015 1000785   2.0795665 -1.6660082  -1.7316055 3.2368336  2.776892   3.293872 0 0 0
2019 1000785   3.0445225 -1.8773173           .  3.257635 3.0021615  3.3603754 0 1 0
2010 1000800    3.091088 -2.0402207           .  3.060952  3.850153   2.856589 0 0 0
2015 1000800   2.9957824 -1.8578993           . 3.3791904  3.480748  2.9033976 0 0 0
2019 1000800    2.564949 -2.1119647           .  3.429331  3.351482    2.90411 0 1 0
2010 1000838   1.3865443  -3.218876           . 3.1153874  2.737794   1.623335 1 0 0
2015 1000838     .693647  -3.101093           .  2.886531 2.2426229   1.729352 1 0 0
2019 1000838    1.609438  -3.352407           .  3.089314 2.5408144  1.6889882 1 1 1
2010 1000849   2.1973357  -3.218876           . 3.3611736  3.734698   2.332438 0 0 0
2015 1000849 .0009995003  -3.270169           . 2.8269584  4.011416  1.9902004 0 0 0
2019 1000849           . -3.5404594           .   2.92531  3.326366   2.409824 0 1 0
2010 1000853   2.9444916  -1.966113    -.701304   3.29059  4.117253   1.943384 0 0 0
2015 1000853    1.946053  -1.609438   -.6792443  3.415166  3.999081    2.78606 0 0 0
2019 1000853    2.564949 -1.7316055           . 3.0543785   3.61135  2.4094644 0 1 0
2010 1000854    3.828663   -2.65926    -.388408  3.507774 3.8010404   2.527176 0 0 0
2015 1000854   3.6376126  -2.501036   -3.473768  3.450559  4.028187  1.9070208 0 0 0
2019 1000854   4.0943446 -2.7968814           . 3.2691126  3.624581   1.943765 0 1 0
2010 1000858   4.2195225 -1.7719568    -.406554 3.3766844   3.68607  2.7844024 0 0 0
2015 1000858   4.0943613 -1.5005835  -.50916034  3.381301  3.500198  2.6047616 0 0 0
2019 1000858   4.0073333 -1.8018098  -1.1147417 3.2271614 3.3270125  2.5703194 0 1 0
2010 1000868   2.0795665 -1.7719568  -.24025233  3.693513   3.22969   3.001002 0 0 0
2015 1000868    1.946053 -1.4481697   -.3368723  3.675338  3.847314   3.248512 0 0 0
2019 1000868   2.6390574 -1.7147985   -.7940731  3.451066  3.951186   2.888481 0 1 0
2010 1000876    2.708117 -2.1202636   -.3045105 3.4470296  3.274551    2.72247 0 0 0
2015 1000876   1.3865443 -1.6347557    .0610951 3.2346704  3.592451  2.5838466 0 0 0
2019 1000876    2.772589 -1.9241487   -.8347107 3.2867215   3.27605   2.731376 0 1 0
2010 1000891    2.397986  -1.469676    .4677448 3.4522765  3.454404   3.051243 1 0 0
2015 1000891    2.639129  -1.398367    .6081338  3.567107  3.462543  3.5628626 1 0 0
2019 1000891    2.484907 -1.6607312   -.7011793  3.264996 3.0020125   3.386219 1 1 1
2010 1000901     2.48499 -1.8325815   -.4795946  3.304786  3.388032  3.1704135 0 0 0
2015 1000901   2.1973357 -1.6554818     -.47965  3.405256    3.4829   3.193271 0 0 0
2019 1000901    3.178054   -1.86433           .    3.1254  3.083972   3.227518 0 1 0
2010 1000902    2.639129  -1.966113  -.23494403  3.433662  3.658579   2.446538 0 0 0
end

Please let me know if this is accessible (this is my first time using dataex, so I could well have got it wrong!)

For clarification, log variables are used to correct for heteroscedasticity.
My research is looking at the effect of youth club closure on youth crime at the neighbourhood level.

Many thanks for your help

[Jared Greathouse]

I'll look at this when I get up in a few minutes, but in the meantime, can you rerun your matching analysis and generate the weights and stuff? And then, edit your data example such that it includes the ps and weight variables?

[Thomas Wake]

Thanks a lot!

Matching code used: psmatch2 Defunded LSOA logemploy loghealth loghousing logenviron logeducation, out(logcrime) common

* Example generated by -dataex-. For more info, type help dataex
clear
input int Year long LSOA float(logcrime logemploy loghealth loghousing logenviron logeducation) byte(Defunded Y2018 Defunded2018) double _pscore byte(_treated _support) double _logcrime int(_id _n1) float _nn double _pdif
2010 1000019   2.1973357 -2.1202636   -.5915127  3.245968 3.5058186   3.419787 0 0 0 .19336741000644028 0 1                  .  71   . 0                     .
2015 1000019    2.302685 -1.6820086   -1.937942  3.437047  3.020961  3.2949846 0 0 0 .13984295022272947 0 1                  .  35   . 0                     .
2019 1000019   2.3025851 -1.9805017   -.7402388 3.7196026 3.2893715   3.262357 0 1 0  .2749901353253076 0 1                  . 120   . 0                     .
2010 1000030    3.496538   -1.89712   -.5987332  3.666462 4.1158667  3.5705876 0 0 0 .32763237939633283 0 1                  . 167   . 0                     .
2015 1000030     3.04457 -1.7092582   -.8580218 3.7568884   3.08571  3.6532004 0 0 0  .2826788302883505 0 1                  . 122   . 0                     .
2019 1000030    3.295837  -1.820159  -4.1351666 3.7975545  3.044189   3.657208 0 1 0  .3331955060491285 0 1                  . 174   . 0                     .
2010 1000051     3.33224 -1.5141277  -.04472836  3.473627 3.9952366   3.563232 0 0 0 .18057091671345565 0 1                  .  62   . 0                     .
2015 1000051    1.946053 -1.6296406   -.6217572  3.874529 3.3255396   3.390406 0 0 0 .27120690068646003 0 1                  . 116   . 0                     .
2019 1000051   1.3862944 -2.0402207   -.9390477 3.8285544  3.314949  3.3392155 0 1 0   .343197118606112 0 1                  . 184   . 0                     .
2010 1000093   2.9957824  -1.609438 -.007344606   3.38233  3.720582  3.5884836 0 0 0  .1691463646091739 0 1                  .  52   . 0                     .
2015 1000093    2.833272 -1.4961092  -.25360277  3.674882 3.2388744   3.432599 0 0 0  .1886583399100983 0 1                  .  67   . 0                     .
2019 1000093   3.2580965 -1.7778566  -.17197527 3.9051394  3.774369   3.330596 0 1 0 .32648736074397805 0 1                  . 165   . 0                     .
2010 1000181   2.0795665  -2.813411           .  3.022832  3.783135 -.01958769 0 0 0                  . . .                  . 384   . .                     .
2015 1000181    1.946053  -3.101093           .  3.045522   3.47822 -1.0106014 0 0 0                  . . .                  . 385   . .                     .
2019 1000181   2.0794415 -3.2441936           .  3.035866  3.390271  -1.171183 0 1 0                  . . .                  . 386   . .                     .
2010 1000259    2.397986  -2.813411           .  3.247443 2.5852005   .6677217 0 0 0                  . . .                  . 387   . .                     .
2015 1000259    2.302685  -2.688248           .  3.412236 3.0348086  1.3972343 0 0 0                  . . .                  . 388   . .                     .
2019 1000259    .6931472 -2.7181005           .   3.40943  2.955587  1.3630258 0 1 0                  . . .                  . 389   . .                     .
2010 1000328   1.0989456  -2.995732           .  2.608232  2.604541  2.6191075 1 0 0                  . . .                  . 390   . .                     .
2015 1000328    2.397986  -2.813411           .  2.904603 2.1170986    2.82684 1 0 0                  . . .                  . 391   . .                     .
2019 1000328    2.833213  -3.015935           . 3.0136704 2.6092606  2.8455486 1 1 1                  . . .                  . 392   . .                     .
2010 1000386   2.1973357 -1.7719568  -.13005367  3.194772  3.802041  3.6577716 1 0 0 .17167779844069872 1 1  3.806684732437134 272  54 1 .00043344534136768864
2015 1000386   2.1973357  -1.639897   -.4215945  3.403694  3.125883   3.685825 1 0 0 .18339849538370262 1 1 2.8332133293151855 276  64 1   .002448427501380912
2019 1000386    2.890372 -1.6347557  -.10869942  3.605253 3.1062446  3.7067194 1 1 1 .24081396529517604 1 1 1.0986123085021973 287  98 1  .0007484142542666938
2010 1000401    2.639129  -2.207275  -1.6966878  2.952555  3.259934    3.56215 0 0 0 .15991904541293298 0 1                  .  49   . 0                     .
2015 1000401    1.791926 -1.9449106   -1.324259  3.280836   3.01067   3.323704 0 0 0 .15850610336152118 0 1                  .  48   . 0                     .
2019 1000401    2.564949 -2.3330443           .  3.240089   3.23589    3.25948 0 1 0                  . . .                  . 393   . .                     .
2010 1000432     2.48499 -2.3025851           .   2.77004  2.857501  3.7460005 1 0 0                  . . .                  . 394   . .                     .
2015 1000432    2.397986 -1.8773173           .  3.341801 2.4803965   3.568828 1 0 0                  . . .                  . 395   . .                     .
2019 1000432   3.3322046  -2.198225           . 3.3792584  2.836795  3.3195534 1 1 1                  . . .                  . 396   . .                     .
2010 1000460 .0009995003  -3.506558           .  3.014804  2.426584  1.5087794 0 0 0                  . . .                  . 397   . .                     .
2015 1000460     .693647  -3.381395           .  2.847406  1.516445  2.0636933 0 0 0                  . . .                  . 398   . .                     .
2019 1000460           0  -3.473768           . 3.2076936  2.083309  2.0882773 0 1 0                  . . .                  . 399   . .                     .
2010 1000483   2.9957824 -1.8325815   -.4675037  3.736555  3.881937   2.905882 1 0 0 .23698185802368943 1 1  2.890427350997925 286  96 1  .0010472851907517289
2015 1000483     2.48499 -1.7897614  -1.2765435  3.707505   3.46411   3.041518 1 0 0  .2256452934214888 1 1  2.890427350997925 285  94 1   .003019920207930654
2019 1000483    3.178054  -1.937942           .  3.828446  3.217795   3.052302 1 1 1                  . . .                  . 400   . .                     .
2010 1000552     2.48499 -1.5606477  -.19332665    3.7224 4.1885056   2.938907 1 0 0 .19813459979876918 1 1 1.6096378564834595 280  76 1 .00033928597344090594
2015 1000552   2.9444916 -1.3056364    -.614336 3.7916164  3.922132  3.2569804 1 0 0 .20455414701298594 1 1  2.484989881515503 281  79 1  .0005011892805886764
2019 1000552    3.178054 -1.6660082   -4.342806  3.745165 3.8357465   3.188499 1 1 1 .26187890565475414 1 1 2.9957823753356934 294 110 1  .0006494698671591359
2010 1000630   2.9444916 -1.8325815   -.8903957 3.7389994  3.667776  2.4547894 1 0 0  .1741358443001812 1 1 3.6376125812530518 273  58 1  .0003022765375091663
2015 1000630    2.833272 -1.9519283           . 3.8343856  3.146692  2.7752104 1 0 0                  . . .                  . 401   . .                     .
2019 1000630    3.583519  -2.225624           .  3.939813    3.1062   2.844153 1 1 1                  . . .                  . 402   . .                     .
2010 1000636   3.5835466 -2.4079456           .  3.621258  3.253313   2.684745 1 0 0                  . . .                  . 403   . .                     .
2015 1000636   2.1973357 -2.4079456           . 3.5182986  3.440322   2.788339 1 0 0                  . . .                  . 404   . .                     .
2019 1000636    2.944439  -2.847312           .  3.653874 3.5704055  2.9325795 1 1 1                  . . .                  . 405   . .                     .
2010 1000643   3.4340196  -1.469676   .09680906  3.720435  3.720909  2.7000046 0 0 0 .14017563539278594 0 1                  .  36   . 0                     .
2015 1000643   2.9957824 -1.3205066    .2350721   3.71557  3.600048  3.1875916 0 0 0 .16787865418176195 0 1                  .  51   . 0                     .
2019 1000643    3.583519  -1.469676  -.16723593  3.861971 3.4727426  3.0207176 0 1 0 .20792148519521386 0 1                  .  82   . 0                     .
2010 1000664    1.609638   -2.65926           .  3.503659 1.9960095  2.6748056 1 0 0                  . . .                  . 406   . .                     .
2015 1000664    2.397986 -2.2926347           .  3.622713 2.2168093    2.79783 1 0 0                  . . .                  . 407   . .                     .
2019 1000664    2.484907  -2.617296           .  3.739668  2.491634    2.75887 1 1 1                  . . .                  . 408   . .                     .
2010 1000677    3.218916  -3.218876           .  3.487832 3.5798976  1.5263253 1 0 0                  . . .                  . 409   . .                     .
2015 1000677    2.833272   -2.97593           . 3.3857114  3.361729   1.399704 1 0 0                  . . .                  . 410   . .                     .
2019 1000677    2.995732  -3.411248           . 3.4682016  3.443682  1.2781522 1 1 1                  . . .                  . 411   . .                     .
2010 1000721    2.772651   -1.89712    -.430555 3.1713636   3.03152  3.8144796 1 0 0 .19413626004388673 1 1  2.302685022354126 278  72 1 .00020406400798200774
2015 1000721    2.639129  -1.461018   -.2823629  3.588617 2.5740616   3.545529 1 0 0 .17437874231193745 1 1 3.6376125812530518 274  58 1  .0005451745492654203
2019 1000721    3.367296 -1.6873995  -.57447565 3.4531255 2.8383765    3.71839 1 1 1 .21275324351583008 1 1  2.079566478729248 282  87 1 .00013784429854743485
2010 1000725    1.946053 -2.3025851           . 3.0567706    3.0288   3.215815 1 0 0                  . . .                  . 412   . .                     .
2015 1000725    2.890427 -2.0174062           .  3.453379 3.1733344   3.209068 1 0 0                  . . .                  . 413   . .                     .
2019 1000725     2.70805 -2.1628232   -3.101093  3.428326   3.27918   3.149054 1 1 1 .24240015704188614 1 1 1.0986123085021973 288  98 1  .0008377774924434034
2010 1000735   2.9444916 -2.0402207  -.43735865  3.069533  3.045417   3.762105 0 0 0 .18697655986564854 0 1                  .  66   . 0                     .
2015 1000735    1.791926 -1.7259717    -1.89712 3.4369826  2.520756  3.8016276 0 0 0 .21989585783955942 0 1                  .  93   . 0                     .
2019 1000735    2.890372 -2.0714734   -6.214608  3.444065  3.157468    3.67663 0 1 0  .3195684419473491 0 1                  . 158   . 0                     .
2010 1000750    2.397986 -1.8325815   -.7946257 3.3559694 1.0582086   3.210722 0 0 0 .10191056826207087 0 1                  .  14   . 0                     .
2015 1000750    2.302685  -1.639897  -1.2039728 3.7184625   1.59615  3.4300115 0 0 0 .19445159368839374 0 1                  .  73   . 0                     .
2019 1000750    2.890372 -1.7897614           .  3.818218  2.123578   3.375469 0 1 0                  . . .                  . 414   . .                     .
2010 1000785    2.890427 -1.8325815    -.633911  2.942722  2.860839   3.766999 0 0 0 .12202474447516044 0 1                  .  20   . 0                     .
2015 1000785   2.0795665 -1.6660082  -1.7316055 3.2368336  2.776892   3.293872 0 0 0  .1115153548671828 0 1                  .  17   . 0                     .
2019 1000785   3.0445225 -1.8773173           .  3.257635 3.0021615  3.3603754 0 1 0                  . . .                  . 415   . .                     .
2010 1000800    3.091088 -2.0402207           .  3.060952  3.850153   2.856589 0 0 0                  . . .                  . 416   . .                     .
2015 1000800   2.9957824 -1.8578993           . 3.3791904  3.480748  2.9033976 0 0 0                  . . .                  . 417   . .                     .
2019 1000800    2.564949 -2.1119647           .  3.429331  3.351482    2.90411 0 1 0                  . . .                  . 418   . .                     .
2010 1000838   1.3865443  -3.218876           . 3.1153874  2.737794   1.623335 1 0 0                  . . .                  . 419   . .                     .
2015 1000838     .693647  -3.101093           .  2.886531 2.2426229   1.729352 1 0 0                  . . .                  . 420   . .                     .
2019 1000838    1.609438  -3.352407           .  3.089314 2.5408144  1.6889882 1 1 1                  . . .                  . 421   . .                     .
2010 1000849   2.1973357  -3.218876           . 3.3611736  3.734698   2.332438 0 0 0                  . . .                  . 422   . .                     .
2015 1000849 .0009995003  -3.270169           . 2.8269584  4.011416  1.9902004 0 0 0                  . . .                  . 423   . .                     .
2019 1000849           . -3.5404594           .   2.92531  3.326366   2.409824 0 1 0                  . . .                  . 424   . .                     .
2010 1000853   2.9444916  -1.966113    -.701304   3.29059  4.117253   1.943384 0 0 0 .07329687468261234 0 1                  .   6   . 0                     .
2015 1000853    1.946053  -1.609438   -.6792443  3.415166  3.999081    2.78606 0 0 0 .12082471199518172 0 1                  .  19   . 0                     .
2019 1000853    2.564949 -1.7316055           . 3.0543785   3.61135  2.4094644 0 1 0                  . . .                  . 425   . .                     .
2010 1000854    3.828663   -2.65926    -.388408  3.507774 3.8010404   2.527176 0 0 0 .29614458625633044 0 1                  . 135   . 0                     .
2015 1000854   3.6376126  -2.501036   -3.473768  3.450559  4.028187  1.9070208 0 0 0 .17383356776267203 0 1                  .  58   . 0                     .
2019 1000854   4.0943446 -2.7968814           . 3.2691126  3.624581   1.943765 0 1 0                  . . .                  . 426   . .                     .
2010 1000858   4.2195225 -1.7719568    -.406554 3.3766844   3.68607  2.7844024 0 0 0  .1262663388301357 0 1                  .  25   . 0                     .
2015 1000858   4.0943613 -1.5005835  -.50916034  3.381301  3.500198  2.6047616 0 0 0 .07610911048370606 0 1                  .   7   . 0                     .
2019 1000858   4.0073333 -1.8018098  -1.1147417 3.2271614 3.3270125  2.5703194 0 1 0 .07845587277173613 0 1                  .  10   . 0                     .
2010 1000868   2.0795665 -1.7719568  -.24025233  3.693513   3.22969   3.001002 0 0 0   .215478033551196 0 1                  .  91   . 0                     .
2015 1000868    1.946053 -1.4481697   -.3368723  3.675338  3.847314   3.248512 0 0 0 .20659174978616385 0 1                  .  81   . 0                     .
2019 1000868   2.6390574 -1.7147985   -.7940731  3.451066  3.951186   2.888481 0 1 0  .1542457376286119 0 1                  .  44   . 0                     .
2010 1000876    2.708117 -2.1202636   -.3045105 3.4470296  3.274551    2.72247 0 0 0 .17727645731451985 0 1                  .  60   . 0                     .
2015 1000876   1.3865443 -1.6347557    .0610951 3.2346704  3.592451  2.5838466 0 0 0 .06841693855034102 0 1                  .   4   . 0                     .
2019 1000876    2.772589 -1.9241487   -.8347107 3.2867215   3.27605   2.731376 0 1 0  .1143196622690931 0 1                  .  18   . 0                     .
2010 1000891    2.397986  -1.469676    .4677448 3.4522765  3.454404   3.051243 1 0 0 .11982441387551618 1 1 1.9460530281066895 266  19 1   .001000298119665538
2015 1000891    2.639129  -1.398367    .6081338  3.567107  3.462543  3.5628626 1 0 0 .19463844253487972 1 1  2.302685022354126 279  73 1 .00018684884648598188
2019 1000891    2.484907 -1.6607312   -.7011793  3.264996 3.0020125   3.386219 1 1 1 .13224795776585632 1 1 1.9460530281066895 267  30 1   .000856819640540657
2010 1000901     2.48499 -1.8325815   -.4795946  3.304786  3.388032  3.1704135 0 0 0 .15432382859480367 0 1                  .  45   . 0                     .
2015 1000901   2.1973357 -1.6554818     -.47965  3.405256    3.4829   3.193271 0 0 0 .15530779561271862 0 1                  .  46   . 0                     .
2019 1000901    3.178054   -1.86433           .    3.1254  3.083972   3.227518 0 1 0                  . . .                  . 427   . .                     .
2010 1000902    2.639129  -1.966113  -.23494403  3.433662  3.658579   2.446538 0 0 0  .1306242487778127 0 1                  .  28   . 0                     .
end
label values _treated _treated
label def _treated 0 "Untreated", modify
label def _treated 1 "Treated", modify
label values _support _support
label def _support 1 "On support", modify

[Jared Greathouse]

Okay, so looking at my homework notes, the code for using your matching within the context of DD is something like

xtdidregress (logcrime)(Defunded) [aw=_weight], group(LSOA) time(Year)

Underneath the hood, what you're doing is you're using xtreg with time and unit fixed effects to re-weight your dataset. In this case, you don't need any covariates, assuming your matching procedure gives you a well balanced sample.

[Thomas Wake]

Thank you for your response, I really appreciate you taking the time to help.

I have now run that, but unfortunately I get the error that the weight must be constant within LSOA. Do you have any advice on how to fix this?

More broadly, how necessary is it that I use matching for my analysis in this context? If I were just to use xtreg and xtdidreg without matching, could I still rely on my results?

Again, thank you very much for your help, it has been invaluable.

[Jared Greathouse]

Oh yeah I forgot this is panel data. In this case, what you're looking for is this command, the flexpaneldid command. It implements what you're looking for.

Using matching is good when you've got many units and few time periods (say, 5 years before a policy and 5 years after) and 2000 units (for example). That's the framework that matching is designed for, many units, fewer time periods. Basic DD means that you need parallel trends, no heterogenous treatment effects. If you can argue that, if you can make the case for that statistically or otherwise, you'll generally be fine.

This brings me to your first post: you say your trends aren't parallel. How can you tell?

[Thomas Wake]

Great, thanks Jared!

I can tell by just looking graphically at the control and treatment.,
I used the code:

[CODE]
collapse (mean) logcrime, by(Defunded Year)
reshape wide logcrime, i(Year) j(Defunded)
graph twoway connect logcrime* Year if Year < 2018
restore
[CODE]

This showed that the log crime lines are not parallel. As an aside, when I do this analysis for Crime (not logged) the lines are parallel - does this have any relevance?

[Øyvind Snilsberg]

As an aside, when I do this analysis for Crime (not logged) the lines are parallel - does this have any relevance?

In general, the common trend assumption depends on the scale of measurement of the outcome variable. Do you need to log transform crime?

[Thomas Wake]

I believe I do have to log Crime. When not logged, my crime variable does not follow a normal distribution

[Maxence Morlet]

Great, thanks Jared!

I can tell by just looking graphically at the control and treatment.,
I used the code:

[CODE]
collapse (mean) logcrime, by(Defunded Year)
reshape wide logcrime, i(Year) j(Defunded)
graph twoway connect logcrime* Year if Year < 2018
restore
[CODE]

This showed that the log crime lines are not parallel. As an aside, when I do this analysis for Crime (not logged) the lines are parallel - does this have any relevance?

Would you mind perhaps showing us the graph? For the entire sample period as well (I presume your intervention time is 2018, correct?)

You may want to run

xtreg [outcome] [controls] i.time#i.treated, fe cluster(clustvar)

and see if the coefficients on the interaction terms are significant for the years pre-intervention.

Actually, Jared Greathouse once recommended on another thread that I read the review of the DiD literature by Roth et al. (2022), a great read I must say and it contains a great discussion of making the PTA pluasible and potential tests to increase its plausibility.

[Thomas Wake]

Would you mind perhaps showing us the graph? For the entire sample period as well (I presume your intervention time is 2018, correct?)

You may want to run

xtreg [outcome] [controls] i.time#i.treated, fe cluster(clustvar)

and see if the coefficients on the interaction terms are significant for the years pre-intervention.

Actually, Jared Greathouse once recommended on another thread that I read the review of the DiD literature by Roth et al. (2022), a great read I must say and it contains a great discussion of making the PTA pluasible and potential tests to increase its plausibility.

I have just run that command and none of the controls come back as significant? What does this mean for my analysis? for context, I am controlling for income, employment, health, education, and housing quality. These have all had to be logged to mitigate heteroscedasticity.

Unfortunately I do not know how to attach the graphs, however, I am able to run

estat ptrends

This returns an F stat greater than 0.1, with H0 being that the trends are parallel - can I use this as a solid enough indicator of the parallel trends test instead?

Appreciate all the help!

[Maxence Morlet]

This certainly works in your favour; you have made the PTA more plausible in your analysis.

I presume you're an economist; economists care a lot about the story you build to convince them of parallel trends, and rely a lot as well of graphical inspection. So now that you've provided "numerical" support in favour of the PTA, if I were I would now focus on building a convincing story of why the policy was not driven by the outcome, is plausibly exogenous, and that were was no selection into treatment base don differential trends.

[Thomas Wake]

Thank you everyone for all your help, it is much appreciated