[Anil Rup]

Thanks Fernando - do you know under what name these (Tmx) estimates are stored in the post estimation? I can't seem to find them. Alternatively, will this command line (estat pretrend, window(-9 9)) will give me the same thing? Many thanks.

[FernandoRios]

that is the name!

here an example that uses event aggregates for the pre-trend test

frause mpdta
csdid  lemp lpop , ivar(countyreal) time(year) gvar(first_treat) method(dripw) long2
estat event, post
test Tm4 Tm3 Tm2

[Anil Rup]

Thank you, Fernando. It worked!

[Alejandro Baron]

Dear Fernando, I hope you are well. I am trying to conduct a DiD using csdid with several HDI subindicators at a subnational level (shdi) as outcome variables (also with several controls). The units in this case are subnational governments (named states/provinces/regions, depending on the country).

When including lead variables to check for how treatment impact next year's outcome, the output flags that the "Panel is not balanced. Will use observations with Pair balanced (observed at t0 and t1)". However, and in contrast with previous posts I have read where this happens, it seems that all computations are successful - no "x" nor "omitted" results in form of zeros in this case. Questions:

- Could the unbalanced part come from the fact that there is no lead data for the last treatment year? (I.e., there are treated units in 2021 but no data for 2022 for the outcome var.) If not, any clue of what could cause it?
- Also, how important is the fact that the panel is not balanced in this situation, if there are no omissions?

Sorry if there are any form-related mistakes in this post - it's my first one on Statalist. Thank you beforehand for your help!

 
*Info on treatment* 

. tab year0 treatyear

           |                       treatyear
     year0 |         0       2015       2017       2019       2021 |     Total
-----------+-------------------------------------------------------+----------
      2011 |     2,706          8          4         24          8 |     2,750 
      2012 |     2,706          8          4         24          8 |     2,750 
      2013 |     2,706          8          4         24          8 |     2,750 
      2014 |     2,706          8          4         24          8 |     2,750 
      2015 |     2,706          8          4         24          8 |     2,750 
      2016 |     2,706          8          4         24          8 |     2,750 
      2017 |     2,706          8          4         24          8 |     2,750 
      2018 |     2,706          8          4         24          8 |     2,750 
      2019 |     2,706          8          4         24          8 |     2,750 
      2020 |     2,706          8          4         24          8 |     2,750 
      2021 |     2,706          8          4         24          8 |     2,750 
-----------+-------------------------------------------------------+----------
     Total |    29,766         88         44        264         88 |    30,250 
--

*Output 1. Year on year -> "balanced"*

. csdid shdiincome shdieduc shdihealth wbcorruption wbvoiceaccount wbruleoflaw wbregulatoryqu wbpoliticalsta wbg
> overneffectiv, ivar(snunitbaseid) time(year0) gvar(treatyear) method (dripw)
........................................
Difference-in-difference with Multiple Time Periods

                                                Number of obs     =     29,986
Outcome model  : least squares
Treatment model: inverse probability
------------------------------------------------------------------------------
             |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
g2015        |
 t_2011_2012 |  -.0008958   .0009657    -0.93   0.354    -.0027886     .000997
 t_2012_2013 |  -.0004298   .0014789    -0.29   0.771    -.0033284    .0024687
 t_2013_2014 |   .0004643   .0006043     0.77   0.442    -.0007201    .0016486
 t_2014_2015 |  -.0000116   .0011759    -0.01   0.992    -.0023162    .0022931
 t_2014_2016 |   .0015915   .0027364     0.58   0.561    -.0037719    .0069548
 t_2014_2017 |   .0008024   .0025529     0.31   0.753    -.0042012     .005806
 t_2014_2018 |  -.0001436   .0028235    -0.05   0.959    -.0056775    .0053903
 t_2014_2019 |  -.0004185   .0027718    -0.15   0.880     -.005851    .0050141
 t_2014_2020 |  -.0025498   .0042858    -0.59   0.552    -.0109498    .0058502
 t_2014_2021 |  -.0003039   .0045829    -0.07   0.947    -.0092862    .0086785
-------------+----------------------------------------------------------------
g2017        |
 t_2011_2012 |   .0035435   .0034346     1.03   0.302    -.0031883    .0102753
 t_2012_2013 |  -.0006468   .0010965    -0.59   0.555     -.002796    .0015024
 t_2013_2014 |   .0027931   .0005839     4.78   0.000     .0016488    .0039375
 t_2014_2015 |   .0028916   .0022557     1.28   0.200    -.0015295    .0073128
 t_2015_2016 |   -.000462   .0033246    -0.14   0.889    -.0069781    .0060541
 t_2016_2017 |  -.0019466   .0022829    -0.85   0.394     -.006421    .0025278
 t_2016_2018 |  -.0047202   .0033924    -1.39   0.164    -.0113692    .0019288
 t_2016_2019 |  -.0054718   .0036728    -1.49   0.136    -.0126704    .0017269
 t_2016_2020 |  -.0085118   .0046759    -1.82   0.069    -.0176765    .0006528
 t_2016_2021 |  -.0081663   .0047191    -1.73   0.084    -.0174155    .0010829
-------------+----------------------------------------------------------------
g2019        |
 t_2011_2012 |   -.002439   .0007598    -3.21   0.001    -.0039283   -.0009497
 t_2012_2013 |  -.0014021   .0011725    -1.20   0.232    -.0037002    .0008961
 t_2013_2014 |   .0000539   .0008269     0.07   0.948    -.0015669    .0016747
 t_2014_2015 |   .0005799   .0008489     0.68   0.495    -.0010839    .0022437
 t_2015_2016 |  -.0044062   .0023311    -1.89   0.059     -.008975    .0001626
 t_2016_2017 |  -.0015747   .0007772    -2.03   0.043    -.0030979   -.0000514
 t_2017_2018 |  -.0019445   .0007534    -2.58   0.010    -.0034211    -.000468
 t_2018_2019 |  -.0026516   .0006893    -3.85   0.000    -.0040026   -.0013006
 t_2018_2020 |    -.00519   .0015868    -3.27   0.001       -.0083     -.00208
 t_2018_2021 |  -.0051591   .0019356    -2.67   0.008    -.0089528   -.0013653
-------------+----------------------------------------------------------------
g2021        |
 t_2011_2012 |  -.0010288   .0013788    -0.75   0.456    -.0037312    .0016735
 t_2012_2013 |  -.0001726   .0021452    -0.08   0.936    -.0043771    .0040318
 t_2013_2014 |   .0004466   .0016992     0.26   0.793    -.0028839     .003777
 t_2014_2015 |   .0037767   .0022892     1.65   0.099    -.0007101    .0082635
 t_2015_2016 |  -.0033758    .002911    -1.16   0.246    -.0090812    .0023296
 t_2016_2017 |   .0018736   .0006848     2.74   0.006     .0005315    .0032156
 t_2017_2018 |   .0000931   .0006521     0.14   0.886    -.0011849    .0013711
 t_2018_2019 |   .0015844   .0013412     1.18   0.237    -.0010443    .0042131
 t_2019_2020 |  -.0033734   .0020234    -1.67   0.095    -.0073391    .0005923
 t_2020_2021 |   .0006507     .00154     0.42   0.673    -.0023677    .0036691
------------------------------------------------------------------------------
Control: Never Treated

See Callaway and Sant'Anna (2021) for details
--

*Output 2. Lead outcome var on year -> "unbalanced"*

. csdid leadshdiincome shdieduc shdihealth wbcorruption wbvoiceaccount wbruleoflaw wbregulatoryqu wbpoliticalsta
>  wbgoverneffectiv, ivar(snunitbaseid) time(year0) gvar(treatyear) method (dripw)
Panel is not balanced
Will use observations with Pair balanced (observed at t0 and t1)
........................................
Difference-in-difference with Multiple Time Periods

                                                Number of obs     =     29,984
Outcome model  : least squares
Treatment model: inverse probability
------------------------------------------------------------------------------
             |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
g2015        |
 t_2011_2012 |  -.0026424   .0007705    -3.43   0.001    -.0041526   -.0011321
 t_2012_2013 |   .0026733   .0018171     1.47   0.141    -.0008882    .0062348
 t_2013_2014 |   .0000373   .0014209     0.03   0.979    -.0027475    .0028222
 t_2014_2015 |   .0018905   .0021959     0.86   0.389    -.0024133    .0061943
 t_2014_2016 |   .0006719   .0019231     0.35   0.727    -.0030973     .004441
 t_2014_2017 |   .0005918   .0021442     0.28   0.783    -.0036107    .0047944
 t_2014_2018 |  -.0000393    .002909    -0.01   0.989    -.0057408    .0056623
 t_2014_2019 |  -.0043089   .0045894    -0.94   0.348    -.0133039    .0046862
 t_2014_2020 |   .0010876   .0044959     0.24   0.809    -.0077242    .0098994
 t_2014_2021 |   .0471884   .0299946     1.57   0.116    -.0115999    .1059767
-------------+----------------------------------------------------------------
g2017        |
 t_2011_2012 |   .0013538   .0018124     0.75   0.455    -.0021985    .0049061
 t_2012_2013 |   .0007278   .0017618     0.41   0.680    -.0027252    .0041808
 t_2013_2014 |   .0028491    .001349     2.11   0.035     .0002051    .0054931
 t_2014_2015 |   .0001916   .0038709     0.05   0.961    -.0073953    .0077784
 t_2015_2016 |  -.0004557    .005287    -0.09   0.931    -.0108181    .0099067
 t_2016_2017 |  -.0033418   .0027915    -1.20   0.231     -.008813    .0021294
 t_2016_2018 |  -.0038901   .0016158    -2.41   0.016    -.0070571   -.0007231
 t_2016_2019 |  -.0073476   .0012618    -5.82   0.000    -.0098206   -.0048745
 t_2016_2020 |  -.0077747   .0013539    -5.74   0.000    -.0104284   -.0051211
 t_2016_2021 |   .0482866   .0534547     0.90   0.366    -.0564827     .153056
-------------+----------------------------------------------------------------
g2019        |
 t_2011_2012 |   -.000523   .0014649    -0.36   0.721    -.0033942    .0023481
 t_2012_2013 |  -.0012557   .0010172    -1.23   0.217    -.0032494    .0007381
 t_2013_2014 |  -.0001475   .0013609    -0.11   0.914    -.0028149    .0025198
 t_2014_2015 |  -.0031005    .002631    -1.18   0.239    -.0082572    .0020563
 t_2015_2016 |  -.0033767   .0009291    -3.63   0.000    -.0051977   -.0015557
 t_2016_2017 |  -.0021926   .0009744    -2.25   0.024    -.0041023   -.0002829
 t_2017_2018 |  -.0013337   .0012514    -1.07   0.287    -.0037865     .001119
 t_2018_2019 |  -.0035461   .0013984    -2.54   0.011    -.0062868   -.0008054
 t_2018_2020 |  -.0021181   .0016243    -1.30   0.192    -.0053017    .0010655
 t_2018_2021 |   -.012627   .0177776    -0.71   0.478    -.0474705    .0222165
-------------+----------------------------------------------------------------
g2021        |
 t_2011_2012 |   .0028646   .0023276     1.23   0.218    -.0016975    .0074267
 t_2012_2013 |  -.0006549   .0019068    -0.34   0.731    -.0043922    .0030823
 t_2013_2014 |   .0038956   .0023324     1.67   0.095    -.0006758     .008467
 t_2014_2015 |  -.0041042   .0036475    -1.13   0.261    -.0112532    .0030449
 t_2015_2016 |  -.0001208   .0017278    -0.07   0.944    -.0035073    .0032657
 t_2016_2017 |   .0035413    .001387     2.55   0.011     .0008229    .0062598
 t_2017_2018 |   .0000603   .0015594     0.04   0.969    -.0029961    .0031166
 t_2018_2019 |  -.0034649   .0027965    -1.24   0.215    -.0089458    .0020161
 t_2019_2020 |   .0001907   .0023988     0.08   0.937    -.0045109    .0048922
 t_2020_2021 |   .0351544   .0344939     1.02   0.308    -.0324524    .1027613
------------------------------------------------------------------------------
Control: Never Treated

See Callaway and Sant'Anna (2021) for details

[Øyvind Snilsberg]

Hi Fernando,

How to manually estimate ATTGT's using Outcome regression DiD (reg) estimation method when covariates are specified?

When no covariates are specified, I can manually estimate ATTGT's. For example, g2004 t_2003_2005:

. use https://friosavila.github.io/playingwithstata/drdid/mpdta.dta, clear
(Written by R.              )

. csdid lemp, ivar(countyreal) time(year) gvar(first_treat) method(reg) notyet
............
Difference-in-difference with Multiple Time Periods

                                                         Number of obs = 2,500
Outcome model  : regression adjustment
Treatment model: none
------------------------------------------------------------------------------
             | Coefficient  Std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
g2004        |
 t_2003_2004 |  -.0193724   .0223101    -0.87   0.385    -.0630994    .0243547
 t_2003_2005 |  -.0783191   .0303902    -2.58   0.010    -.1378829   -.0187553
 t_2003_2006 |  -.1362743   .0354034    -3.85   0.000    -.2056637    -.066885
 t_2003_2007 |  -.1008114   .0343592    -2.93   0.003    -.1681542   -.0334685
-------------+----------------------------------------------------------------
g2006        |
 t_2003_2004 |  -.0025626   .0225302    -0.11   0.909     -.046721    .0415959
 t_2004_2005 |  -.0019392   .0190422    -0.10   0.919    -.0392612    .0353827
 t_2005_2006 |   .0046609   .0163356     0.29   0.775    -.0273563     .036678
 t_2005_2007 |  -.0412245   .0202292    -2.04   0.042    -.0808729    -.001576
-------------+----------------------------------------------------------------
g2007        |
 t_2003_2004 |   .0305067   .0150336     2.03   0.042     .0010414    .0599719
 t_2004_2005 |  -.0027259   .0163958    -0.17   0.868    -.0348611    .0294093
 t_2005_2006 |  -.0310871   .0178775    -1.74   0.082    -.0661264    .0039522
 t_2006_2007 |  -.0260544   .0166554    -1.56   0.118    -.0586985    .0065896
------------------------------------------------------------------------------
Control: Not yet Treated

See Callaway and Sant'Anna (2021) for details

. tab first_treat, gen(group)

first.treat |      Freq.     Percent        Cum.
------------+-----------------------------------
          0 |      1,545       61.80       61.80
       2004 |        100        4.00       65.80
       2006 |        200        8.00       73.80
       2007 |        655       26.20      100.00
------------+-----------------------------------
      Total |      2,500      100.00

. tab year, gen(time)

       year |      Freq.     Percent        Cum.
------------+-----------------------------------
       2003 |        500       20.00       20.00
       2004 |        500       20.00       40.00
       2005 |        500       20.00       60.00
       2006 |        500       20.00       80.00
       2007 |        500       20.00      100.00
------------+-----------------------------------
      Total |      2,500      100.00

. reg lemp group2##time3 if inlist(year,2003,2005) & inlist(first_treat,0,2004,2006,2007), nohe
------------------------------------------------------------------------------
        lemp | Coefficient  Std. err.      t    P>|t|     [95% conf. interval]
-------------+----------------------------------------------------------------
    1.group2 |   .3970694   .3430753     1.16   0.247     -.276164    1.070303
     1.time3 |  -.0419256   .0970364    -0.43   0.666    -.2323448    .1484935
             |
group2#time3 |
        1 1  |  -.0783191   .4851818    -0.16   0.872    -1.030415    .8737767
             |
       _cons |   5.782627   .0686151    84.28   0.000     5.647981    5.917274
------------------------------------------------------------------------------

. 
end of do-file

How to manually estimate g2004 t_2003_2005 when specifying covariate lpop?

. use https://friosavila.github.io/playingwithstata/drdid/mpdta.dta, clear
(Written by R.              )

. csdid lemp lpop, ivar(countyreal) time(year) gvar(first_treat) method(reg) notyet
............
Difference-in-difference with Multiple Time Periods

                                                         Number of obs = 2,500
Outcome model  : regression adjustment
Treatment model: none
------------------------------------------------------------------------------
             | Coefficient  Std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
g2004        |
 t_2003_2004 |   -.021248   .0216163    -0.98   0.326    -.0636153    .0211193
 t_2003_2005 |    -.08185   .0281112    -2.91   0.004     -.136947    -.026753
 t_2003_2006 |   -.138469   .0338809    -4.09   0.000    -.2048744   -.0720636
 t_2003_2007 |  -.1075443   .0327377    -3.29   0.001     -.171709   -.0433796
-------------+----------------------------------------------------------------
g2006        |
 t_2003_2004 |  -.0080821   .0217655    -0.37   0.710    -.0507416    .0345775
 t_2004_2005 |  -.0062169   .0180305    -0.34   0.730     -.041556    .0291222
 t_2005_2006 |   .0093754    .016906     0.55   0.579    -.0237598    .0425106
 t_2005_2007 |  -.0415356   .0197169    -2.11   0.035      -.08018   -.0028913
-------------+----------------------------------------------------------------
g2007        |
 t_2003_2004 |   .0263658   .0140189     1.88   0.060    -.0011108    .0538425
 t_2004_2005 |  -.0047598     .01567    -0.30   0.761    -.0354724    .0259527
 t_2005_2006 |  -.0285021   .0181321    -1.57   0.116    -.0640403    .0070361
 t_2006_2007 |  -.0287895   .0161679    -1.78   0.075    -.0604779    .0028989
------------------------------------------------------------------------------
Control: Not yet Treated

See Callaway and Sant'Anna (2021) for details

. 
end of do-file

Thanks,
Øyvind

[FernandoRios]

While it is possible to do it using regression (like what you get using jwdid), you can it as follows

frause mpdta, clear
keep if inlist(year,2003,2005) 
bysort countyreal (year):gen dlemp = lemp[2]-lemp[1]
reg dlemp lpop if first_treat!=2004
predict hdlemp
gen ddlemp = dlemp - hdlemp
reg ddlemp if first==2004

[Alejandro Baron]

Hi FernandoRios, I was wondering if you had any advice regarding entry #349 of this thread that I posted a few days ago. Thank you beforehand for your advice and help.

[FernandoRios]

Hi there
yes if there are no leads or lags (nature of the data) some observations will not be fully observed in 2x2 did
now that is just consequence of data missing. I don’t think it would lead to any problems
also be aware that you are using too
many controls
your effective sample at the lowest is 2. So adding controls may not be a good idea
Fernando

[Alejandro Baron]

Thank you Fernando!

[Andre Dupont]

Does this command work only in the newer versions of STATA? I'm trying to run it with Stata 13 and it gives me an error code.
Thanks.

[FernandoRios]

Unfortunately only with Stata 14 or above

[Andre Dupont]

Unfortunately only with Stata 14 or above

Thank you.

[Andre Dupont]

I'm running the csdid command on a dataset with various options, such as different methods (dripw, reg, ipw) or with the notyet option included or not, and I get exactly the same results for all of these options. Does this make sense? From the help file, I would have expected somewhat different results.

[FernandoRios]

If you have no controls, all should give you the same results.

[Kal El]

Is there a way to get the csdid_plot command to plot the reference point (0 difference) like a normal event study plot?