Basing stars on non-standard p-values in estout

[Mitch Downey]

Hello,

I'm using some "custom" block-bootstrap code to calculate standard errors for a range of regression results. I'd like to display these results (coefficients, standard errors, and stars representing p-values) in a table using something like estout (which I like very much). I've figured out how to have estout display the bootstrapped standard errors, but I can't figure out how to change the p-values on which it is basing the stars. Here's an example:

sysuse auto, clear

* The regression of interest:
local vlist mpg trunk
reg price `vlist'
estimates store e1

* Make a matrix with bootstrapped standard errors.
* In reality, this is based on bootstrapped results (not just invented numbers), but this is a simplification.
matrix define bsemat = J(1,3,0)
matrix colnames bsemat = "`vlist' _cons"
matrix bsemat[1,1] = 1.1
matrix bsemat[1,2] = 2.2
matrix bsemat[1,3] = 0.3

* Adding my standard errors to the estimation results.
estimates restore e1
estadd matrix seboot = bsemat

* Displaying the results with "real" standard errors:
estout e1, cells(b(star fmt(%9.3f)) se(par fmt(%9.3f))) starlevels(* 0.10 ** 0.05 *** 0.01) stats(N r2, fmt(0 3))
* Displaying the results with my standard errors:
estout e1, cells(b(star fmt(%9.3f)) seboot(par fmt(%9.3f))) starlevels(* 0.10 ** 0.05 *** 0.01) stats(N r2, fmt(0 3))

I'm happy with how easy it is to have estout display my standard errors rather than the true ones, but obviously it has no idea that they're standard errors. I don't know how to adjust the input that stars is using. Should I overwrite the information in e(V)? If so, how?

Two quick comments:
1. I'm using an estimation procedure that doesn't allow me to simply use "vce(bootstrap)".
2. I'm certainly open to suggestions on the overall process that I'm using.

Thanks,
Mitch

[David Weiss]

I'm curious if anyone has an answer to this question

[Nils Enevoldsen]

Apologies for necroing a five year old thread, but I recently encountered this. A solution is below, and changes I made are bolded and in red. I changed the bootstrapped se matrix numbers for legibility. You don't need to modify e(V), but you do need to estadd a matrix of p values. Your method of computing that matrix depends on your assumptions about your computed variance. Here, I assumed normality. You could use, e.g., ttest() instead of normal() if your assumptions are different.

sysuse auto, clear

* The regression of interest:
local vlist mpg trunk
reg price `vlist'
estimates store e1

* Make a matrix with bootstrapped standard errors.
* In reality, this is based on bootstrapped results (not just invented numbers), but this is a simplification.
matrix define bsemat = J(1,3,0)
matrix colnames bsemat = "`vlist' _cons"
matrix bsemat[1,1] = 120
matrix bsemat[1,2] = 20
matrix bsemat[1,3] = 1000

* Adding my standard errors to the estimation results.
estimates restore e1
estadd matrix seboot = bsemat

* Adding my p values to the estimation results.
matrix define betamat = e(b)
matrix define bpmat = J(1,3,0)
matrix colnames bpmat = "`vlist' _cons"
forvalues i = 1/1 {
    forvalues j = 1/3 {
         matrix bpmat[`i',`j']=2*(normal(-abs(betamat[`i',`j']/bsemat[`i',`j'])))
    }
}
estadd matrix pboot = bpmat

* Displaying the results with "real" standard errors:
estout e1, cells(b(star fmt(%9.3f)) se(par fmt(%9.3f))) starlevels(* 0.10 ** 0.05 *** 0.01) stats(N r2, fmt(0 3))

* Displaying the results with my standard errors:
estout e1, cells(b(star pvalue(pboot) fmt(%9.3f)) seboot(par fmt(%9.3f))) starlevels(* 0.10 ** 0.05 *** 0.01) stats(N r2, fmt(0 3))

[Anzony Quispe]

You do not know how much I appreciate you have answered this question. Thanks a lot!!

[Syed Abul Hasan]

Hi Nils,
I also find the codes very useful in adding my randomised inference test statistics (ritest). The only issue is that the stars are not appearing with the esttab command. I wonder whether you experience the same and can suggest anything? My modified codes are given below.
Kind regards, Syed

sysuse auto, clear
matrix drop _all
foreach c in price weight {
* The regression of interest:
local vlist mpg trunk
reg `c' `vlist'
estimates store e`c'

* Generating bootstrapped se
bootstrap: reg `c' `vlist'

* Make a matrix with bootstrapped standard errors.
matrix define bsemat = (_se[mpg])
matrix colnames bsemat = "mpg"

* Adding my standard errors to the estimation results.
estimates restore e`c'
estadd matrix seboot = bsemat

* Adding my p values to the estimation results.
matrix define betamat = e(b)
// matrix define bpmat = (2*(normal(abs(betamat[1,1]/bsemat[1,1]))))
matrix define bpmat = (2 * ttail(e(df_r), abs(betamat[1,1]/bsemat[1,1])))
matrix colnames bpmat = "mpg"
estadd matrix pboot = bpmat
mat list bpmat
}
* Displaying the results with my standard errors:
esttab eprice eweight, cells(b(star pvalue(pboot) fmt(%9.3f)) seboot(par fmt(%9.3f))) starlevels(* 0.10 ** 0.05 *** 0.01) keep(mpg) noobs nonumber nomtitle

[Nils Enevoldsen]

Hi Syed,

I didn't investigate in detail, but when I run your code I do get stars. Do you not get stars? If not, what if you restart Stata?

For reference, my versions are:

. which estout
/Users/nils/Library/Application Support/Stata/ado/plus/e/estout.ado
*! version 3.24  30apr2021  Ben Jann

. which esttab
/Users/nils/Library/Application Support/Stata/ado/plus/e/esttab.ado
*! version 2.1.0  19may2021  Ben Jann
*! wrapper for estout

. which estadd
/Users/nils/Library/Application Support/Stata/ado/plus/e/estadd.ado
*! version 2.3.5  05feb2016  Ben Jann

. version
version 16.1

. sysuse auto, clear
(1978 Automobile Data)

. matrix drop _all

. foreach c in price weight {
  2. * The regression of interest:
. local vlist mpg trunk
  3. reg `c' `vlist'
  4. estimates store e`c'
  5. 
. * Generating bootstrapped se
. bootstrap: reg `c' `vlist'
  6. 
. * Make a matrix with bootstrapped standard errors.
. matrix define bsemat = (_se[mpg])
  7. matrix colnames bsemat = "mpg"
  8. 
. * Adding my standard errors to the estimation results.
. estimates restore e`c'
  9. estadd matrix seboot = bsemat
 10. 
. * Adding my p values to the estimation results.
. matrix define betamat = e(b)
 11. // matrix define bpmat = (2*(normal(abs(betamat[1,1]/bsemat[1,1]))))
. matrix define bpmat = (2 * ttail(e(df_r), abs(betamat[1,1]/bsemat[1,1])))
 12. matrix colnames bpmat = "mpg"
 13. estadd matrix pboot = bpmat
 14. mat list bpmat
 15. }

      Source |       SS           df       MS      Number of obs   =        74
-------------+----------------------------------   F(2, 71)        =     10.14
       Model |   141126459         2  70563229.4   Prob > F        =    0.0001
    Residual |   493938937        71  6956886.44   R-squared       =    0.2222
-------------+----------------------------------   Adj R-squared   =    0.2003
       Total |   635065396        73  8699525.97   Root MSE        =    2637.6

------------------------------------------------------------------------------
       price |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         mpg |  -220.1649   65.59262    -3.36   0.001    -350.9529    -89.3769
       trunk |   43.55851   88.71884     0.49   0.625    -133.3418    220.4589
       _cons |   10254.95   2349.084     4.37   0.000      5571.01    14938.89
------------------------------------------------------------------------------
(running regress on estimation sample)

Bootstrap replications (50)
----+--- 1 ---+--- 2 ---+--- 3 ---+--- 4 ---+--- 5 
..................................................    50

Linear regression                               Number of obs     =         74
                                                Replications      =         50
                                                Wald chi2(2)      =      20.14
                                                Prob > chi2       =     0.0000
                                                R-squared         =     0.2222
                                                Adj R-squared     =     0.2003
                                                Root MSE          =  2637.5910

------------------------------------------------------------------------------
             |   Observed   Bootstrap                         Normal-based
       price |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         mpg |  -220.1649    77.9294    -2.83   0.005    -372.9037   -67.42607
       trunk |   43.55851   80.76242     0.54   0.590    -114.7329    201.8499
       _cons |   10254.95   2692.185     3.81   0.000     4978.363    15531.54
------------------------------------------------------------------------------
(results eprice are active now)

added matrix:
             e(seboot) :  1 x 1

added matrix:
              e(pboot) :  1 x 1

symmetric bpmat[1,1]
          mpg
r1  .00613084

      Source |       SS           df       MS      Number of obs   =        74
-------------+----------------------------------   F(2, 71)        =     88.48
       Model |  31468208.8         2  15734104.4   Prob > F        =    0.0000
    Residual |  12625969.6        71  177830.557   R-squared       =    0.7137
-------------+----------------------------------   Adj R-squared   =    0.7056
       Total |  44094178.4        73  604029.841   Root MSE        =     421.7

------------------------------------------------------------------------------
      weight |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         mpg |   -84.4932   10.48699    -8.06   0.000    -105.4037   -63.58274
       trunk |   55.67266   14.18442     3.92   0.000     27.38973    83.95559
       _cons |   4053.061   375.5729    10.79   0.000      3304.19    4801.932
------------------------------------------------------------------------------
(running regress on estimation sample)

Bootstrap replications (50)
----+--- 1 ---+--- 2 ---+--- 3 ---+--- 4 ---+--- 5 
..................................................    50

Linear regression                               Number of obs     =         74
                                                Replications      =         50
                                                Wald chi2(2)      =     344.01
                                                Prob > chi2       =     0.0000
                                                R-squared         =     0.7137
                                                Adj R-squared     =     0.7056
                                                Root MSE          =   421.6996

------------------------------------------------------------------------------
             |   Observed   Bootstrap                         Normal-based
      weight |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         mpg |   -84.4932   9.456297    -8.94   0.000    -103.0272    -65.9592
       trunk |   55.67266   11.60731     4.80   0.000     32.92275    78.42257
       _cons |   4053.061   327.1369    12.39   0.000     3411.885    4694.238
------------------------------------------------------------------------------
(results eweight are active now)

added matrix:
             e(seboot) :  1 x 1

added matrix:
              e(pboot) :  1 x 1

symmetric bpmat[1,1]
          mpg
r1  3.094e-13

. * Displaying the results with my standard errors:
. esttab eprice eweight, cells(b(star pvalue(pboot) fmt(%9.3f)) seboot(par fmt(%9.3f))) starlevels(*
>  0.10 ** 0.05 *** 0.01) keep(mpg) noobs nonumber nomtitle

--------------------------------------------
                 b/seboot        b/seboot   
--------------------------------------------
mpg              -220.165***      -84.493***
                 (77.929)         (9.456)   
--------------------------------------------

. 
end of do-file

. do "/var/folders/fw/2__b11qn2q325jd54x5861pm0000gp/T//SD77462.000000"

. sysuse auto, clear
(1978 Automobile Data)

. matrix drop _all

. foreach c in price weight {
  2.         * The regression of interest:
.         local vlist mpg trunk
  3.         reg `c' `vlist'
  4.         estimates store e`c'
  5. 
.         * Generating bootstrapped se
.         bootstrap: reg `c' `vlist'
  6. 
.         * Make a matrix with bootstrapped standard errors.
.         matrix define bsemat = (_se[mpg])
  7.         matrix colnames bsemat = "mpg"
  8. 
.         * Adding my standard errors to the estimation results.
.         estimates restore e`c'
  9.         estadd matrix seboot = bsemat
 10. 
.         * Adding my p values to the estimation results.
.         matrix define betamat = e(b)
 11.         // matrix define bpmat = (2*(normal(abs(betamat[1,1]/bsemat[1,1]))))
.         matrix define bpmat = (2 * ttail(e(df_r), abs(betamat[1,1]/bsemat[1,1])))
 12.         matrix colnames bpmat = "mpg"
 13.         estadd matrix pboot = bpmat
 14.         mat list bpmat
 15. }

      Source |       SS           df       MS      Number of obs   =        74
-------------+----------------------------------   F(2, 71)        =     10.14
       Model |   141126459         2  70563229.4   Prob > F        =    0.0001
    Residual |   493938937        71  6956886.44   R-squared       =    0.2222
-------------+----------------------------------   Adj R-squared   =    0.2003
       Total |   635065396        73  8699525.97   Root MSE        =    2637.6

------------------------------------------------------------------------------
       price |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         mpg |  -220.1649   65.59262    -3.36   0.001    -350.9529    -89.3769
       trunk |   43.55851   88.71884     0.49   0.625    -133.3418    220.4589
       _cons |   10254.95   2349.084     4.37   0.000      5571.01    14938.89
------------------------------------------------------------------------------
(running regress on estimation sample)

Bootstrap replications (50)
----+--- 1 ---+--- 2 ---+--- 3 ---+--- 4 ---+--- 5 
..................................................    50

Linear regression                               Number of obs     =         74
                                                Replications      =         50
                                                Wald chi2(2)      =      29.89
                                                Prob > chi2       =     0.0000
                                                R-squared         =     0.2222
                                                Adj R-squared     =     0.2003
                                                Root MSE          =  2637.5910

------------------------------------------------------------------------------
             |   Observed   Bootstrap                         Normal-based
       price |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         mpg |  -220.1649   72.10551    -3.05   0.002    -361.4891   -78.84068
       trunk |   43.55851   68.25355     0.64   0.523      -90.216     177.333
       _cons |   10254.95   2479.771     4.14   0.000     5394.688    15115.21
------------------------------------------------------------------------------
(results eprice are active now)

added matrix:
             e(seboot) :  1 x 1

added matrix:
              e(pboot) :  1 x 1

symmetric bpmat[1,1]
          mpg
r1  .00318396

      Source |       SS           df       MS      Number of obs   =        74
-------------+----------------------------------   F(2, 71)        =     88.48
       Model |  31468208.8         2  15734104.4   Prob > F        =    0.0000
    Residual |  12625969.6        71  177830.557   R-squared       =    0.7137
-------------+----------------------------------   Adj R-squared   =    0.7056
       Total |  44094178.4        73  604029.841   Root MSE        =     421.7

------------------------------------------------------------------------------
      weight |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         mpg |   -84.4932   10.48699    -8.06   0.000    -105.4037   -63.58274
       trunk |   55.67266   14.18442     3.92   0.000     27.38973    83.95559
       _cons |   4053.061   375.5729    10.79   0.000      3304.19    4801.932
------------------------------------------------------------------------------
(running regress on estimation sample)

Bootstrap replications (50)
----+--- 1 ---+--- 2 ---+--- 3 ---+--- 4 ---+--- 5 
..................................................    50

Linear regression                               Number of obs     =         74
                                                Replications      =         50
                                                Wald chi2(2)      =     175.36
                                                Prob > chi2       =     0.0000
                                                R-squared         =     0.7137
                                                Adj R-squared     =     0.7056
                                                Root MSE          =   421.6996

------------------------------------------------------------------------------
             |   Observed   Bootstrap                         Normal-based
      weight |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         mpg |   -84.4932   12.55024    -6.73   0.000    -109.0912   -59.89518
       trunk |   55.67266   12.31562     4.52   0.000     31.53449    79.81083
       _cons |   4053.061   388.7235    10.43   0.000     3291.177    4814.945
------------------------------------------------------------------------------
(results eweight are active now)

added matrix:
             e(seboot) :  1 x 1

added matrix:
              e(pboot) :  1 x 1

symmetric bpmat[1,1]
          mpg
r1  3.621e-09

. * Displaying the results with my standard errors:
. esttab eprice eweight, cells(b(star pvalue(pboot) fmt(%9.3f)) pboot(par fmt(%9.3f)))
>  starlevels(* 0.10 ** 0.05 *** 0.01) keep(mpg) noobs nonumber nomtitle

--------------------------------------------
                  b/pboot         b/pboot   
--------------------------------------------
mpg              -220.165***      -84.493***
                  (0.003)         (0.000)   
--------------------------------------------

[Syed Abul Hasan]

Hi Nils, You correctly guessed the issue; I was using old versions of estadd, esttab and estout. The updated package is now providing the significance stars in tex output. Thanks very much for your help. Kind regards, Syed

[Benjamin Alvarez]

Hi everyone!

I have a matrix of corrected p-values (different from the ones from the regression) and I want them to appear in the table that I am exporting (with the command estout). I want them to appear in () below the coefficients.

How can I do this?


Thanks you very much!

[Nils Enevoldsen]

If your p-values are in yourmatrix, then you can display them below the coefficients in parentheses with:

cells(b yourmatrix(par))

If you additionally want significance stars, then:

cells(b(star pvalue(yourmatrix)) yourmatrix(par))

edit: You must estadd the matrix to your estimation results first, and the column names of the matrix must be specified appropriately, as seen above.