How to read the result of the generalized test by using DiD following Callaway and Sant'Anna (2020)?

Phuc Nguyen

Join Date: Mar 2017
Posts: 348

How to read the result of the generalized test by using DiD following Callaway and Sant'Anna (2020)?

01 Sep 2021, 14:54

I am using the generalized DiD following [Callaway and Sant'Anna (2020)] by applying the package

Code:

csdid

in STATA to examine whether the staggered implementation laws have impact on a specific dependent variable Y.
My code is

Code:

gen gvar = cond(ei==., 0, ei) // group variable as required for the csdid command

 csdid y $control_list , ivar(i) time(t) gvar(gvar) notyet

And I got the result as below, but I do not know where to read the coefficient of variable of interest (whether the laws have impact on dependent y) based on the results shown

Code:

Panel is not balanced
Will use observations with Pair balanced (observed at t0 and t1)
xxxxxxxxxxxxxxxxxxxxxxx..x....xxxxxxxxxxxxxxxx..x.
...xxxxxxxxxxxxxxxxx..x...xxxxxxxxxxxxxxxx........
.xx..xxxxxxxxxxxxxx.x.xxxxxxxxxxxxxxxx........x.xx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.xx..xxxxxxxxxxx
xxxxxxxxx.xxxxxxxx..xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx..xxxxxxxxxxxx
Difference-in-difference with Multiple Time Periods
Outcome model  : 
Treatment model: 
------------------------------------------------------------------------------
             | Coefficient  Std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
g1993        |
 t_1990_1991 |          0  (omitted)
 t_1991_1992 |          0  (omitted)
 t_1992_1993 |          0  (omitted)
 t_1992_1994 |          0  (omitted)
 t_1992_1995 |          0  (omitted)
 t_1992_1996 |          0  (omitted)
 t_1992_1997 |          0  (omitted)
 t_1992_1998 |          0  (omitted)
 t_1992_1999 |          0  (omitted)
 t_1992_2000 |          0  (omitted)
 t_1992_2001 |          0  (omitted)
 t_1992_2002 |          0  (omitted)
 t_1992_2003 |          0  (omitted)
 t_1992_2004 |          0  (omitted)
 t_1992_2005 |          0  (omitted)
 t_1992_2006 |          0  (omitted)
 t_1992_2007 |          0  (omitted)
 t_1992_2008 |          0  (omitted)
 t_1992_2009 |          0  (omitted)
 t_1992_2010 |          0  (omitted)
 t_1992_2011 |          0  (omitted)
 t_1992_2012 |          0  (omitted)
 t_1992_2013 |          0  (omitted)
-------------+----------------------------------------------------------------
g1998        |
 t_1990_1991 |   .0044891   .0155562     0.29   0.773    -.0260005    .0349788
 t_1991_1992 |  -.0446835   .0164184    -2.72   0.006     -.076863    -.012504
 t_1992_1993 |          0  (omitted)
 t_1993_1994 |   .0078568   .0187939     0.42   0.676    -.0289787    .0446922
 t_1994_1995 |   .0275184   .0167182     1.65   0.100    -.0052486    .0602855
 t_1995_1996 |   .0603187   .0130332     4.63   0.000     .0347741    .0858634
 t_1996_1997 |     -.0154   .0154548    -1.00   0.319    -.0456908    .0148908
 t_1997_1998 |          0  (omitted)
 t_1997_1999 |          0  (omitted)
 t_1997_2000 |          0  (omitted)
 t_1997_2001 |          0  (omitted)
 t_1997_2002 |          0  (omitted)
 t_1997_2003 |          0  (omitted)
 t_1997_2004 |          0  (omitted)
 t_1997_2005 |          0  (omitted)
 t_1997_2006 |          0  (omitted)
 t_1997_2007 |          0  (omitted)
 t_1997_2008 |          0  (omitted)
 t_1997_2009 |          0  (omitted)
 t_1997_2010 |          0  (omitted)
 t_1997_2011 |          0  (omitted)
 t_1997_2012 |          0  (omitted)
 t_1997_2013 |          0  (omitted)
-------------+----------------------------------------------------------------
g2000        |
 t_1990_1991 |  -.0055218   .0152681    -0.36   0.718    -.0354468    .0244033
 t_1991_1992 |  -.0125322   .0148237    -0.85   0.398     -.041586    .0165217
 t_1992_1993 |          0  (omitted)
 t_1993_1994 |   .0315272   .0135471     2.33   0.020     .0049755     .058079
 t_1994_1995 |  -.0044671   .0117418    -0.38   0.704    -.0274806    .0185465
 t_1995_1996 |  -.0257615   .0107184    -2.40   0.016    -.0467693   -.0047538
 t_1996_1997 |   .0003434   .0122374     0.03   0.978    -.0236415    .0243283
 t_1997_1998 |          0  (omitted)
 t_1998_1999 |          0  (omitted)
 t_1999_2000 |          0  (omitted)
 t_1999_2001 |          0  (omitted)
 t_1999_2002 |          0  (omitted)
 t_1999_2003 |          0  (omitted)
 t_1999_2004 |          0  (omitted)
 t_1999_2005 |          0  (omitted)
 t_1999_2006 |          0  (omitted)
 t_1999_2007 |          0  (omitted)
 t_1999_2008 |          0  (omitted)
 t_1999_2009 |          0  (omitted)
 t_1999_2010 |          0  (omitted)
 t_1999_2011 |          0  (omitted)
 t_1999_2012 |          0  (omitted)
 t_1999_2013 |          0  (omitted)
-------------+----------------------------------------------------------------
g2001        |
 t_1990_1991 |          0  (omitted)
 t_1991_1992 |   .0114934   .0140626     0.82   0.414    -.0160687    .0390556
 t_1992_1993 |  -.0263688   .0174505    -1.51   0.131    -.0605712    .0078336
 t_1993_1994 |          0  (omitted)
 t_1994_1995 |   -.022652   .0136074    -1.66   0.096    -.0493221    .0040181
 t_1995_1996 |   .0099844   .0122639     0.81   0.416    -.0140523    .0340212
 t_1996_1997 |  -.0086703   .0168704    -0.51   0.607    -.0417357    .0243951
 t_1997_1998 |          0  (omitted)
 t_1998_1999 |          0  (omitted)
 t_1999_2000 |          0  (omitted)
 t_2000_2001 |          0  (omitted)
 t_2000_2002 |          0  (omitted)
 t_2000_2003 |          0  (omitted)
 t_2000_2004 |          0  (omitted)
 t_2000_2005 |          0  (omitted)
 t_2000_2006 |          0  (omitted)
 t_2000_2007 |          0  (omitted)
 t_2000_2008 |          0  (omitted)
 t_2000_2009 |          0  (omitted)
 t_2000_2010 |          0  (omitted)
 t_2000_2011 |          0  (omitted)
 t_2000_2012 |          0  (omitted)
 t_2000_2013 |          0  (omitted)
-------------+----------------------------------------------------------------
g2002        |
 t_1990_1991 |   .0047549   .0202444     0.23   0.814    -.0349233    .0444332
 t_1991_1992 |   -.000291   .0186843    -0.02   0.988    -.0369115    .0363295
 t_1992_1993 |   .0332336    .021247     1.56   0.118    -.0084097    .0748769
 t_1993_1994 |   .0461132   .0203395     2.27   0.023     .0062485    .0859779
 t_1994_1995 |   .0270379   .0195139     1.39   0.166    -.0112087    .0652845
 t_1995_1996 |  -.0348445   .0179321    -1.94   0.052    -.0699908    .0003019
 t_1996_1997 |   .0078077   .0240225     0.33   0.745    -.0392755     .054891
 t_1997_1998 |   .0009777   .0198905     0.05   0.961     -.038007    .0399623
 t_1998_1999 |   .0297011   .0210341     1.41   0.158    -.0115251    .0709272
 t_1999_2000 |          0  (omitted)
 t_2000_2001 |          0  (omitted)
 t_2001_2002 |  -.0079708   .0358483    -0.22   0.824    -.0782322    .0622906
 t_2001_2003 |   .0704668   .0504999     1.40   0.163    -.0285113    .1694449
 t_2001_2004 |          0  (omitted)
 t_2001_2005 |          0  (omitted)
 t_2001_2006 |          0  (omitted)
 t_2001_2007 |          0  (omitted)
 t_2001_2008 |          0  (omitted)
 t_2001_2009 |          0  (omitted)
 t_2001_2010 |          0  (omitted)
 t_2001_2011 |          0  (omitted)
 t_2001_2012 |          0  (omitted)
 t_2001_2013 |          0  (omitted)
-------------+----------------------------------------------------------------
g2003        |
 t_1990_1991 |          0  (omitted)
 t_1991_1992 |          0  (omitted)
 t_1992_1993 |          0  (omitted)
 t_1993_1994 |          0  (omitted)
 t_1994_1995 |  -.0568093   .0330009    -1.72   0.085    -.1214898    .0078713
 t_1995_1996 |          0  (omitted)
 t_1996_1997 |   .0401619   .0331058     1.21   0.225    -.0247243     .105048
 t_1997_1998 |          0  (omitted)
 t_1998_1999 |          0  (omitted)
 t_1999_2000 |          0  (omitted)
 t_2000_2001 |          0  (omitted)
 t_2001_2002 |          0  (omitted)
 t_2002_2003 |          0  (omitted)
 t_2002_2004 |          0  (omitted)
 t_2002_2005 |          0  (omitted)
 t_2002_2006 |          0  (omitted)
 t_2002_2007 |          0  (omitted)
 t_2002_2008 |          0  (omitted)
 t_2002_2009 |          0  (omitted)
 t_2002_2010 |          0  (omitted)
 t_2002_2011 |          0  (omitted)
 t_2002_2012 |          0  (omitted)
 t_2002_2013 |          0  (omitted)
-------------+----------------------------------------------------------------
g2004        |
 t_1990_1991 |  -.0269685   .0211212    -1.28   0.202    -.0683654    .0144283
 t_1991_1992 |   .0436563   .0219829     1.99   0.047     .0005706     .086742
 t_1992_1993 |  -.0380752   .0196921    -1.93   0.053    -.0766709    .0005205
 t_1993_1994 |  -.0311921    .021651    -1.44   0.150    -.0736273    .0112431
 t_1994_1995 |  -.0210135   .0180164    -1.17   0.243     -.056325     .014298
 t_1995_1996 |    .013557   .0170045     0.80   0.425    -.0197711    .0468852
 t_1996_1997 |   .0366271    .016159     2.27   0.023     .0049559    .0682982
 t_1997_1998 |   .0670258   .0255287     2.63   0.009     .0169903    .1170612
 t_1998_1999 |          0  (omitted)
 t_1999_2000 |   .0721523   .0489685     1.47   0.141    -.0238241    .1681287
 t_2000_2001 |          0  (omitted)
 t_2001_2002 |          0  (omitted)
 t_2002_2003 |          0  (omitted)
 t_2003_2004 |          0  (omitted)
 t_2003_2005 |          0  (omitted)
 t_2003_2006 |          0  (omitted)
 t_2003_2007 |          0  (omitted)
 t_2003_2008 |          0  (omitted)
 t_2003_2009 |          0  (omitted)
 t_2003_2010 |          0  (omitted)
 t_2003_2011 |          0  (omitted)
 t_2003_2012 |          0  (omitted)
 t_2003_2013 |          0  (omitted)
-------------+----------------------------------------------------------------
g2005        |
 t_1990_1991 |          0  (omitted)
 t_1991_1992 |          0  (omitted)
 t_1992_1993 |          0  (omitted)
 t_1993_1994 |          0  (omitted)
 t_1994_1995 |          0  (omitted)
 t_1995_1996 |          0  (omitted)
 t_1996_1997 |          0  (omitted)
 t_1997_1998 |          0  (omitted)
 t_1998_1999 |          0  (omitted)
 t_1999_2000 |          0  (omitted)
 t_2000_2001 |          0  (omitted)
 t_2001_2002 |          0  (omitted)
 t_2002_2003 |          0  (omitted)
 t_2003_2004 |          0  (omitted)
 t_2004_2005 |          0  (omitted)
 t_2004_2006 |          0  (omitted)
 t_2004_2007 |          0  (omitted)
 t_2004_2008 |          0  (omitted)
 t_2004_2009 |          0  (omitted)
 t_2004_2010 |          0  (omitted)
 t_2004_2011 |          0  (omitted)
 t_2004_2012 |          0  (omitted)
 t_2004_2013 |          0  (omitted)
-------------+----------------------------------------------------------------
g2006        |
 t_1990_1991 |  -.0060986   .0551538    -0.11   0.912     -.114198    .1020009
 t_1991_1992 |          0  (omitted)
 t_1992_1993 |          0  (omitted)
 t_1993_1994 |    .000406   .0344024     0.01   0.991    -.0670214    .0678335
 t_1994_1995 |  -.0016141   .0227773    -0.07   0.944    -.0462568    .0430285
 t_1995_1996 |          0  (omitted)
 t_1996_1997 |          0  (omitted)
 t_1997_1998 |          0  (omitted)
 t_1998_1999 |          0  (omitted)
 t_1999_2000 |          0  (omitted)
 t_2000_2001 |          0  (omitted)
 t_2001_2002 |          0  (omitted)
 t_2002_2003 |          0  (omitted)
 t_2003_2004 |          0  (omitted)
 t_2004_2005 |          0  (omitted)
 t_2005_2006 |          0  (omitted)
 t_2005_2007 |          0  (omitted)
 t_2005_2008 |          0  (omitted)
 t_2005_2009 |          0  (omitted)
 t_2005_2010 |          0  (omitted)
 t_2005_2011 |          0  (omitted)
 t_2005_2012 |          0  (omitted)
 t_2005_2013 |          0  (omitted)
-------------+----------------------------------------------------------------
g2007        |
 t_1990_1991 |          0  (omitted)
 t_1991_1992 |          0  (omitted)
 t_1992_1993 |   .0150089   .0183479     0.82   0.413    -.0209522    .0509701
 t_1993_1994 |          0  (omitted)
 t_1994_1995 |          0  (omitted)
 t_1995_1996 |          0  (omitted)
 t_1996_1997 |          0  (omitted)
 t_1997_1998 |          0  (omitted)
 t_1998_1999 |          0  (omitted)
 t_1999_2000 |          0  (omitted)
 t_2000_2001 |          0  (omitted)
 t_2001_2002 |  -.1035312   .0481367    -2.15   0.031    -.1978773    -.009185
 t_2002_2003 |   .0398218   .0329674     1.21   0.227    -.0247931    .1044367
 t_2003_2004 |          0  (omitted)
 t_2004_2005 |          0  (omitted)
 t_2005_2006 |          0  (omitted)
 t_2006_2007 |          0  (omitted)
 t_2006_2008 |          0  (omitted)
 t_2006_2009 |          0  (omitted)
 t_2006_2010 |          0  (omitted)
 t_2006_2011 |          0  (omitted)
 t_2006_2012 |          0  (omitted)
 t_2006_2013 |          0  (omitted)
-------------+----------------------------------------------------------------
g2008        |
 t_1990_1991 |          0  (omitted)
 t_1991_1992 |          0  (omitted)
 t_1992_1993 |          0  (omitted)
 t_1993_1994 |          0  (omitted)
 t_1994_1995 |          0  (omitted)
 t_1995_1996 |          0  (omitted)
 t_1996_1997 |          0  (omitted)
 t_1997_1998 |          0  (omitted)
 t_1998_1999 |          0  (omitted)
 t_1999_2000 |          0  (omitted)
 t_2000_2001 |          0  (omitted)
 t_2001_2002 |          0  (omitted)
 t_2002_2003 |          0  (omitted)
 t_2003_2004 |          0  (omitted)
 t_2004_2005 |          0  (omitted)
 t_2005_2006 |          0  (omitted)
 t_2006_2007 |          0  (omitted)
 t_2007_2008 |          0  (omitted)
 t_2007_2009 |          0  (omitted)
 t_2007_2010 |          0  (omitted)
 t_2007_2011 |          0  (omitted)
 t_2007_2012 |          0  (omitted)
 t_2007_2013 |          0  (omitted)
-------------+----------------------------------------------------------------
g2011        |
 t_1990_1991 |          0  (omitted)
 t_1991_1992 |          0  (omitted)
 t_1992_1993 |          0  (omitted)
 t_1993_1994 |          0  (omitted)
 t_1994_1995 |          0  (omitted)
 t_1995_1996 |          0  (omitted)
 t_1996_1997 |          0  (omitted)
 t_1997_1998 |          0  (omitted)
 t_1998_1999 |          0  (omitted)
 t_1999_2000 |          0  (omitted)
 t_2000_2001 |          0  (omitted)
 t_2001_2002 |          0  (omitted)
 t_2002_2003 |          0  (omitted)
 t_2003_2004 |          0  (omitted)
 t_2004_2005 |          0  (omitted)
 t_2005_2006 |          0  (omitted)
 t_2006_2007 |          0  (omitted)
 t_2007_2008 |          0  (omitted)
 t_2008_2009 |          0  (omitted)
 t_2009_2010 |          0  (omitted)
 t_2010_2011 |          0  (omitted)
 t_2010_2012 |          0  (omitted)
 t_2010_2013 |          0  (omitted)
-------------+----------------------------------------------------------------
g2015        |
 t_1990_1991 |          0  (omitted)
 t_1991_1992 |          0  (omitted)
 t_1992_1993 |          0  (omitted)
 t_1993_1994 |          0  (omitted)
 t_1994_1995 |          0  (omitted)
 t_1995_1996 |          0  (omitted)
 t_1996_1997 |          0  (omitted)
 t_1997_1998 |          0  (omitted)
 t_1998_1999 |          0  (omitted)
 t_1999_2000 |  -.0225901   .0256777    -0.88   0.379    -.0729175    .0277372
 t_2000_2001 |    .001301    .028187     0.05   0.963    -.0539446    .0565466
 t_2001_2002 |          0  (omitted)
 t_2002_2003 |          0  (omitted)
 t_2003_2004 |          0  (omitted)
 t_2004_2005 |          0  (omitted)
 t_2005_2006 |          0  (omitted)
 t_2006_2007 |          0  (omitted)
 t_2007_2008 |          0  (omitted)
 t_2008_2009 |          0  (omitted)
 t_2009_2010 |          0  (omitted)
 t_2010_2011 |          0  (omitted)
 t_2011_2012 |          0  (omitted)
 t_2012_2013 |          0  (omitted)
------------------------------------------------------------------------------
Control: Not yet Treated

Tags: None

George Ford

Join Date: Aug 2014

Posts: 2850
#2

01 Sep 2021, 17:25

try "estat simple" after the model.

Last edited by George Ford; 01 Sep 2021, 17:34.
Comment
George Ford

Join Date: Aug 2014

Posts: 2850
#3

01 Sep 2021, 17:36

and you might want to tend to the failure of almost all of your bootstraps, though I am unfamiliar with csdid
1 like
Comment
FernandoRios

Join Date: Apr 2014

Posts: 2372
#4

01 Sep 2021, 19:52

Dear Phuc
a few clarifications.
1. The "." and "x" do not correspond to bootstrap repetitions but to the 2x2 DID estimations. X's means that particular 2x2 estimation failed.
2. As George Says, you do need to revise your model. My best guess is that you have very few observations for some of the Cohorts, and because of that it the program is failing to get something for many of those iterations.
So,
a) Start by looking at a cross table of Year vs Gvar. That will give you an idea of your control and treatment groups. If you have to many "small" groups, then you need to rethink your approach, or perhaps use alternative estimation methods.
b) Based on number of observations per cell, you need to reevaluate your control variables.
c) After that, you can also try different estimation methods, including -method(reg)-. It is less likely fail.
HTH
1 like
Comment

Announcement

How to read the result of the generalized test by using DiD following Callaway and Sant'Anna (2020)?

Comment

Comment

Comment