significance stars with esttab and contrasts of margins

[Dimitriy V. Masterov]

I am trying to add significance stars to esttab with the following command:

sysuse lifeexp
reg lexp i.region#c.popgrowth 
margins r.region, dydx(popgrowth) contrast(effects) post
esttab, cells(b(star) se) starlevels(+ .3)

To quote David Giles, a table without stars is like champagne without bubbles. How can I add the stars based on the p-values in e(p)?

[Phil Bromiley]

I don't use esttab, but outreg2 handles stars nicely.

Phil

[Amy Ng]

The reason your champagne has no bubbles might be because there's no cork to pop (just yet): the p-values look like they're 0.704 and 0.882!

-----------------------------------------------------------------------------------------
                        |   Contrast Delta-method
                        |      dy/dx   Std. Err.      t    P>|t|     [95% Conf. Interval]
------------------------+----------------------------------------------------------------
popgrowth               |
                 region |
(N.A. vs Eur & C.Asia)  |  -.3740837   .9816405    -0.38   0.704    -2.335136    1.586969
(S.A. vs Eur & C.Asia)  |  -.1513155   1.017362    -0.15   0.882     -2.18373    1.881099
-----------------------------------------------------------------------------------------

[Dimitriy V. Masterov]

That is why I cranked up the alpha to 30% in the esttab. In my own data, the contrast is significant at 1%, but I still see no stars. Unfortunately, I cannot share my data for this example.

[Carlos Zambrana]

Hi,

I ran into this problem and I haven't been able to find a solution. However, I have an example that may give some more credit to Mr. Masterov's (and my own) request for help, and will perhaps shed a bit of light into what's happening.

If I am not mistaken, after running margins, contrast post, the p-values that are posted to e() are associated with the F-statistics (which also include results from the overall test), and esttab doesn't associate them with the coefficients in e(b). Hopefully this example makes it clear:

webuse nlswork, clear

qui: reg ln_wage i.collgrad##(c.ttl_exp##c.ttl_exp##c.ttl_exp c.tenure##c.tenure) i.msp i.race i.ind_code i.occ_code
est sto ln_wagereg

margins, at( ttl_exp=(1 5 10 15 20)) over( i.collgrad) post /*Margins without contrasts*/
est sto marg1

est restore ln_wagereg
margins, at( ttl_exp=(1 5 10 15 20)) over(r.collgrad) contrast(effect) post /*Contrasts w.r.t. non-college graduates*/
est sto marg2

The results from esttab:

. esttab marg1 marg2, label nogaps star(* 0.1 ** 0.05 *** 0.01)

----------------------------------------------------
                              (1)             (2)  
                                                    
----------------------------------------------------
1._at # 1 if colle~0        1.494***                
                         (277.15)                  
1._at # 1 if colle~1        1.890***        0.396  
                         (106.60)         (21.42)  
2._at # 1 if colle~0        1.600***                
                         (436.87)                  
2._at # 1 if colle~1        1.972***        0.373  
                         (234.02)         (40.59)  
3._at # 1 if colle~0        1.681***                
                         (358.64)                  
3._at # 1 if colle~1        2.102***        0.421  
                         (248.26)         (43.58)  
4._at # 1 if colle~0        1.754***                
                         (221.28)                  
4._at # 1 if colle~1        2.222***        0.468  
                         (161.62)         (29.60)  
5._at # 1 if colle~0        1.875***                
                          (96.08)                  
5._at # 1 if colle~1        2.294***        0.419  
                          (50.10)          (8.43)  
c1                                               ***
                                                    
c2                                               ***
                                                    
c3                                               ***
                                                    
c4                                               ***
                                                    
c5                                               ***
                                                    
c6                                               ***
                                                    
----------------------------------------------------
Observations                27662                  
----------------------------------------------------
t statistics in parentheses
* p<0.1, ** p<0.05, *** p<0.01

The p-values, and hence the stars, of estimates posted after contrast seem associated with the coefficients in e(F):

est restore marg2
    
*Print e(b), there are 5 coefficients
matrix list e(b)
    
e(b)[1,5]
     r1vs0.coll~d@  r1vs0.coll~d@  r1vs0.coll~d@  r1vs0.coll~d@  r1vs0.coll~d@
            1._at          2._at          3._at          4._at          5._at
y1      .39633041      .37264637      .42120894      .46831739      .41935703

*Print e(F) and e(p) to see that e(p) and e(F) contain 6 coefficients. If
*I'm not mistaken, the 6th value is from the overall significance test.

matrix list e(p)

e(p)[1,6]
           c1         c2         c3         c4         c5         c6
r1  6.18e-101          0          0  1.22e-189  3.533e-17          0

    matrix list e(F)

e(F)[1,6]
           c1         c2         c3         c4         c5         c6
r1  458.67827  1647.4366  1899.4008  876.39896  71.114731  1150.9829

Please note that the column names are c1 to c6, just like the coefficients that do have stars in the esttab results.

Now ask esttab to report p-values instead of t-statistics:

. esttab marg1 marg2, label nogaps star(* 0.1 ** 0.05 *** 0.01) p

----------------------------------------------------
                              (1)             (2)   
                                                    
----------------------------------------------------
1._at # 1 if colle~0        1.494***                
                          (0.000)                   
1._at # 1 if colle~1        1.890***        0.396   
                          (0.000)                   
2._at # 1 if colle~0        1.600***                
                          (0.000)                   
2._at # 1 if colle~1        1.972***        0.373   
                          (0.000)                   
3._at # 1 if colle~0        1.681***                
                          (0.000)                   
3._at # 1 if colle~1        2.102***        0.421   
                          (0.000)                   
4._at # 1 if colle~0        1.754***                
                          (0.000)                   
4._at # 1 if colle~1        2.222***        0.468   
                          (0.000)                   
5._at # 1 if colle~0        1.875***                
                          (0.000)                   
5._at # 1 if colle~1        2.294***        0.419   
                          (0.000)                   
c1                                               ***
                                          (0.000)   
c2                                               ***
                                          (0.000)   
c3                                               ***
                                          (0.000)   
c4                                               ***
                                          (0.000)   
c5                                               ***
                                          (0.000)   
c6                                               ***
                                          (0.000)   
----------------------------------------------------
Observations                27662                   
----------------------------------------------------
p-values in parentheses
* p<0.1, ** p<0.05, *** p<0.01

No stars or p-values shown for coefficients posted in e(b) after 'contrast', yet the rows for c1 to c6 have stars and p-values but no coefficients!

[Eric de Souza]

A table without stars is like champagne without bubbles

This is quoting Giles out of context:
http://davegiles.blogspot.be/2013/04/star-wars.html
Esttab is user-written (Ben Jann). It may not have been updated by the author.

[Carlos Zambrana]

I tried setting the p-values using the pvalue option of the cells option, but I had no luck. Here's an imperfect workaround:

If your sample size is big enough that you don't mind letting estout or esttab find the p-values and stars assuming Normality or the t-distribution (I am not sure which one it uses), then the short program foo2 in the link below allows you to repost the b and V matrices and clear the rest of the stuff in e(). Use esttab afterwards and it will print stars and p-values nicely, the only problem is that they are not exact. In my data the differences start in the 6th decimal so I don't care, but depending on your data you may want to compare the original p-values that were in e(p) with the ones that esttab gives you.
http://www.stata.com/statalist/archi.../msg00742.html

[Dimitriy V. Masterov]

That's also good enough for my purposes. Thanks, Carlos!

[Alexis Rodas]

I have the same problema, esttab does not display significance stars to margins command
with factorial variables

[Carlos Zambrana]

Try the workaround I wrote up there. It works for me with both esttab and estout2