adding boottest results with estadd scalar

[Sofia Hernandez]

Hello community!

I am running multiple regressions using a for loop and my end result is a table with as many columns as there are regressions (8 columns). I want to store my estimate, standard error, and the p-value from boottest for each regression. I saw a post by David Roodman (creater of boottest comand) on how to use estadd to include the p-value, and adding it via the scalar() option: https://www.statalist.org/forums/for...imation-tables

An example of my code is:

est clear
foreach b in "sampleA" "sampleB" "sampleC" "sampleD"{
foreach s in "maleSample" "femaleSample" {
if "`b'" == "sampleA" {
eststo : qui reg female_w i.PostOct92 $u_p1 if `b'==1 & `s'==1, vce(cluster x)
qui boottest 1.PostOct92, seed(1234) nograph
estadd scalar p = r(p)
}
else {
eststo: qui reg female_w i.PostOct92 $u_p2 if `b'==1 & `s'==1, vce(cluster x)
qui boottest 1.PostOct92, seed(1234) nograph
estadd scalar p = r(p)
}
}
}

esttab * using "new\uc_output_female_w.tex", append f ///
cells(b(fmt(3)) se(fmt(3) star par([ ]))) noobs keep(1.PostOct92) varlabels(1.PostOct92 " ") ///
star(* 0.10 ** 0.05 *** 0.01) alignment(D{.}{.}{-1}) ///
stats(p) ///
label booktabs nomtitle noline nonumber collabels(none) eqlabel(none)

While I can see in the console that the p-value is being added as a scalar after each regression, some other p-values (and I don't know where from) are being added to the table when using esttab. I can see this in the console:

(est1 stored)

added scalar:
                  e(p) =  0

while the p-value being displayed in my .tex table is "0.534". My belief is that this is some stored p-value from some another regression, but it is definitely not the one I want. Would anyone know if I am using the correct command, or possibly overseeing something? Maybe the scalar() option is incorrect?

Any help is appreciated!

Best regards,
Sofia

[David Roodman]

The display code looks right to me. Can you post a minimum working example with actual data, be it yours or a publicly available data set?

See https://www.statalist.org/forums/help#stata

[Sofia Hernandez]

Hi David, thanks for the reply! Here is a snippet of my data:

* Example generated by -dataex-. For more info, type help dataex
clear
input double female_w float PostOct92 double x float x_sq
0 0  -9   81
0 0  -9   81
0 1  15  225
0 0 -21  441
1 0 -21  441
1 1  27  729
1 0 -33 1089
1 1  15  225
1 0  -9   81
1 1  15  225
0 1   3    9
1 0 -21  441
1 1  39 1521
0 1  15  225
0 0 -33 1089
1 0  -9   81
0 0 -21  441
0 0  -9   81
1 0  -9   81
0 0 -21  441
end

Essentially, I am running a parametric RDD where month of birth is my running variable. PostOct92 is a dummy equal to 1 if the individual was born after October 1992, while x and x_sq are polynomials of order 1 and 2 centered around October 1992. My outcome is whether the individual applied to a "typically female" university program (female_w). Running the code I presented above, I obtain these p-values from the boottest:

.

foreach b in "sampleA" "sampleB" "sampleC" "sampleD"{
  2.         foreach s in "maleSample" "femaleSample" {
  3.                 if "`b'" == "sampleA" {
  4.                 eststo : qui reg female_w i.PostOct92 $u_p1 if `b'==1 & `s'==1, vc
> e(cluster x)
  5.                 qui boottest 1.PostOct92, seed(1234) cluster(x) nograph
  6.                 estadd scalar p = r(p)
  7.                 }
  8.                 else {
  9.                 eststo: qui reg female_w i.PostOct92 $u_p2 if `b'==1 & `s'==1, vce
> (cluster x)   
 10.                 qui boottest 1.PostOct92, seed(1234) cluster(x) nograph
 11.                 estadd scalar p = r(p)
 12.                 }       
 13.         }       
 14. }
(est1 stored)

added scalar:
                  e(p) =  0
(est2 stored)

added scalar:
                  e(p) =  .09375
(est3 stored)

added scalar:
                  e(p) =  .98298298
(est4 stored)

added scalar:
                  e(p) =  .2952953
(est5 stored)

added scalar:
                  e(p) =  .03803804
(est6 stored)

added scalar:
                  e(p) =  .88788789
(est7 stored)

added scalar:
                  e(p) =  .00500501
(est8 stored)

added scalar:
                  e(p) =  .3013013

The result from the esttab command is:

.  esttab *, cells(b(fmt(3)) se(fmt(3) star par([ ]))) noobs keep(1.PostOct92) stats(p) 

--------------------------------------------------------------------------------------------------------------------------------------------
                      (1)             (2)             (3)             (4)             (5)             (6)             (7)             (8)   
                 female_w        female_w        female_w        female_w        female_w        female_w        female_w        female_w   
                     b/se            b/se            b/se            b/se            b/se            b/se            b/se            b/se   
--------------------------------------------------------------------------------------------------------------------------------------------
1.PostOct92        -0.013           0.019          -0.000           0.014          -0.014           0.001          -0.017          -0.010   
                  [0.008]         [0.004]**       [0.007]         [0.007]         [0.008]         [0.007]         [0.007]*        [0.009]   
--------------------------------------------------------------------------------------------------------------------------------------------
p                   0.000           0.000           0.000           0.000           0.051           0.638           0.000           0.534   
--------------------------------------------------------------------------------------------------------------------------------------------

where "p" should be the boottest p-value. If you look closely, the p-values do not match. I have trying clearing all estimates, re-importing dataset, re-installing estout; however, I do not understand what p-values the esttab command is displaying.

Thanks for your help!

Best regards,
Sofia

[David Roodman]

I ran that code on that data and got:

sampleA not found

[Sofia Hernandez]

My apologies, I forgot to include those variables. Here is a complete list of the variables used in my estimation:

* Example generated by -dataex-. For more info, type help dataex
clear
input double female_w float PostOct92 double x float(x_sq post_x post_x2 sampleA sampleB sampleC sampleD femaleSample maleSample)
0 0  -9   81  0    0 0 0 1 1 0 1
0 0  -9   81  0    0 0 0 1 1 1 0
0 1  15  225 15  225 0 0 0 0 0 1
0 0 -21  441  0    0 0 0 0 0 1 0
1 0 -21  441  0    0 0 0 0 0 1 0
1 1  27  729 27  729 0 0 0 0 1 0
1 0 -33 1089  0    0 0 0 0 0 1 0
1 1  15  225 15  225 0 0 0 0 1 0
1 0  -9   81  0    0 0 0 1 1 0 1
1 1  15  225 15  225 0 0 0 0 0 1
0 1   3    9  3    9 0 1 1 1 1 0
1 0 -21  441  0    0 0 0 0 0 0 1
1 1  39 1521 39 1521 0 0 0 0 1 0
0 1  15  225 15  225 0 0 0 0 0 1
0 0 -33 1089  0    0 0 0 0 0 1 0
1 0  -9   81  0    0 0 0 1 1 1 0
0 0 -21  441  0    0 0 0 0 0 1 0
0 0  -9   81  0    0 0 0 1 1 1 0
1 0  -9   81  0    0 0 0 1 1 0 1
0 0 -21  441  0    0 0 0 0 0 0 1
end

where I define u_p1 and u_p2

global u_p1 x post_x 
global u_p2 x x_sq post_x post_x2

now the code should work. Thank you!

[David Roodman]

Still not working for me:

. do "C:\Users\drood\AppData\Local\Temp\STDff08_000000.tmp"

. clear

. input double female_w float PostOct92 double x float(x_sq post_x post_x2 sampleA sampleB sampleC sampleD femaleSample maleSample)

       female_w  PostOct92           x       x_sq     post_x    post_x2    sampleA    sampleB    sampleC    sampleD  femaleS~e  maleSam~e
  1. 0 0  -9   81  0    0 0 0 1 1 0 1
  2. 0 0  -9   81  0    0 0 0 1 1 1 0
  3. 0 1  15  225 15  225 0 0 0 0 0 1
  4. 0 0 -21  441  0    0 0 0 0 0 1 0
  5. 1 0 -21  441  0    0 0 0 0 0 1 0
  6. 1 1  27  729 27  729 0 0 0 0 1 0
  7. 1 0 -33 1089  0    0 0 0 0 0 1 0
  8. 1 1  15  225 15  225 0 0 0 0 1 0
  9. 1 0  -9   81  0    0 0 0 1 1 0 1
 10. 1 1  15  225 15  225 0 0 0 0 0 1
 11. 0 1   3    9  3    9 0 1 1 1 1 0
 12. 1 0 -21  441  0    0 0 0 0 0 0 1
 13. 1 1  39 1521 39 1521 0 0 0 0 1 0
 14. 0 1  15  225 15  225 0 0 0 0 0 1
 15. 0 0 -33 1089  0    0 0 0 0 0 1 0
 16. 1 0  -9   81  0    0 0 0 1 1 1 0
 17. 0 0 -21  441  0    0 0 0 0 0 1 0
 18. 0 0  -9   81  0    0 0 0 1 1 1 0
 19. 1 0  -9   81  0    0 0 0 1 1 0 1
 20. 0 0 -21  441  0    0 0 0 0 0 0 1
 21. end

. 
. global u_p1 x post_x 

. global u_p2 x x_sq post_x post_x2

. 
. 
. foreach b in "sampleA" "sampleB" "sampleC" "sampleD"{
  2.          foreach s in "maleSample" "femaleSample" {
  3.                  if "`b'" == "sampleA" {
  4.                  eststo : qui reg female_w i.PostOct92 $u_p1 if `b'==1 & `s'==1, vce(cluster x)
  5.                  qui boottest 1.PostOct92, seed(1234) cluster(x) nograph
  6.                  estadd scalar p = r(p)
  7.                  }
  8.                  else {
  9.                  eststo: qui reg female_w i.PostOct92 $u_p2 if `b'==1 & `s'==1, vce(cluster x)   
 10.                  qui boottest 1.PostOct92, seed(1234) cluster(x) nograph
 11.                  estadd scalar p = r(p)
 12.                  }       
 13.          }       
 14.  }
no observations
r(2000);

[Sofia Hernandez]

Please try this more simple version of the code:

clear
input double female_w float PostOct92 double x float(x_sq post_x post_x2 sampleA sampleB sampleC sampleD)
1 0 -20  400  0    0 1 0 1 1
1 1  21  441 21  441 0 1 1 1
1 0 -22  484  0    0 0 0 0 1
0 1  11  121 11  121 0 0 0 1
1 1   0    0  0    0 1 1 1 1
0 0 -29  841  0    0 0 0 1 0
0 0 -38 1444  0    0 0 1 0 1
0 0 -20  400  0    0 1 0 1 0
0 0 -14  196  0    0 0 0 0 0
0 0 -37 1369  0    0 0 1 1 0
1 1  30  900 30  900 0 0 1 0
0 0 -33 1089  0    0 1 0 0 1
1 0 -40 1600  0    0 0 1 1 1
1 0 -26  676  0    0 0 0 0 1
1 1  24  576 24  576 1 0 1 1
1 0 -29  841  0    0 0 1 0 1
0 0 -17  289  0    0 0 0 0 1
0 0 -36 1296  0    0 0 0 0 1
1 0  -4   16  0    0 1 1 1 1
1 1  36 1296 36 1296 0 1 1 1
0 0 -34 1156  0    0 1 0 0 1
1 1  20  400 20  400 0 1 1 1
0 1  30  900 30  900 0 0 1 0
0 0 -40 1600  0    0 1 0 0 1
1 0 -26  676  0    0 0 1 0 0
0 0 -36 1296  0    0 1 0 1 0
1 0 -36 1296  0    0 0 1 0 1
1 0 -25  625  0    0 0 0 0 1
1 1  33 1089 33 1089 1 1 0 0
0 0 -23  529  0    0 0 0 1 1
0 0 -24  576  0    0 0 1 1 1
1 0 -38 1444  0    0 0 0 1 0
1 1   7   49  7   49 0 0 1 1
0 0 -29  841  0    0 0 1 0 1
1 0 -33 1089  0    0 1 0 0 0
1 0  -4   16  0    0 0 1 1 1
0 0 -16  256  0    0 0 1 0 0
1 0 -40 1600  0    0 0 1 1 1
1 0 -29  841  0    0 0 0 1 0
0 0 -16  256  0    0 0 1 0 1
end

global u_p1 x post_x 
global u_p2 x x_sq post_x post_x2 
 
est clear 
foreach b in "sampleA" "sampleB" "sampleC" "sampleD"{
        eststo : qui reg female_w i.PostOct92 $u_p1 if `b'==1, vce(cluster x)
        qui boottest 1.PostOct92, seed(1234) cluster(x) nograph
        estadd scalar p = r(p)
}       

esttab *, cells(b(fmt(3)) se(fmt(3) star par([ ]))) noobs keep(1.PostOct92) stats(p)

I get the following results in my console:

(est1 stored)

added scalar:
                  e(p) =  .30078125
(est2 stored)

added scalar:
                  e(p) =  .48248248
(est3 stored)

added scalar:
                  e(p) =  .4034034
(est4 stored)

added scalar:
                  e(p) =  .96096096

. 
end of do-file

. do "C:\Users\ssinani\AppData\Local\Temp\4\STDd164_000000.tmp"

. esttab *, cells(b(fmt(3)) se(fmt(3) star par([ ]))) noobs keep(1.PostOct92) stats(p) 

----------------------------------------------------------------------------
                      (1)             (2)             (3)             (4)   
                 female_w        female_w        female_w        female_w   
                     b/se            b/se            b/se            b/se   
----------------------------------------------------------------------------
1.PostOct92        -0.079           0.414           0.371          -0.022   
                  [0.070]         [0.477]         [0.369]         [0.410]   
----------------------------------------------------------------------------
p                       .               .           0.053           0.010   
----------------------------------------------------------------------------

Thanks for the patience - this should work now!

[Andrew Musau]

boottest and estout are from SSC (FAQ Advice #12). As you have experienced, not including a reproducible example reduces your likelihood of obtaining helpful replies. David was patient enough to nudge you to include one. The issue is that you are not storing the statistics with the regression estimates. You need:

clear
input double female_w float PostOct92 double x float(x_sq post_x post_x2 sampleA sampleB sampleC sampleD)
1 0 -20  400  0    0 1 0 1 1
1 1  21  441 21  441 0 1 1 1
1 0 -22  484  0    0 0 0 0 1
0 1  11  121 11  121 0 0 0 1
1 1   0    0  0    0 1 1 1 1
0 0 -29  841  0    0 0 0 1 0
0 0 -38 1444  0    0 0 1 0 1
0 0 -20  400  0    0 1 0 1 0
0 0 -14  196  0    0 0 0 0 0
0 0 -37 1369  0    0 0 1 1 0
1 1  30  900 30  900 0 0 1 0
0 0 -33 1089  0    0 1 0 0 1
1 0 -40 1600  0    0 0 1 1 1
1 0 -26  676  0    0 0 0 0 1
1 1  24  576 24  576 1 0 1 1
1 0 -29  841  0    0 0 1 0 1
0 0 -17  289  0    0 0 0 0 1
0 0 -36 1296  0    0 0 0 0 1
1 0  -4   16  0    0 1 1 1 1
1 1  36 1296 36 1296 0 1 1 1
0 0 -34 1156  0    0 1 0 0 1
1 1  20  400 20  400 0 1 1 1
0 1  30  900 30  900 0 0 1 0
0 0 -40 1600  0    0 1 0 0 1
1 0 -26  676  0    0 0 1 0 0
0 0 -36 1296  0    0 1 0 1 0
1 0 -36 1296  0    0 0 1 0 1
1 0 -25  625  0    0 0 0 0 1
1 1  33 1089 33 1089 1 1 0 0
0 0 -23  529  0    0 0 0 1 1
0 0 -24  576  0    0 0 1 1 1
1 0 -38 1444  0    0 0 0 1 0
1 1   7   49  7   49 0 0 1 1
0 0 -29  841  0    0 0 1 0 1
1 0 -33 1089  0    0 1 0 0 0
1 0  -4   16  0    0 0 1 1 1
0 0 -16  256  0    0 0 1 0 0
1 0 -40 1600  0    0 0 1 1 1
1 0 -29  841  0    0 0 0 1 0
0 0 -16  256  0    0 0 1 0 1
end

global u_p1 x post_x
global u_p2 x x_sq post_x post_x2
 
estimates clear
local i 0
foreach b in "sampleA" "sampleB" "sampleC" "sampleD"{
        local++i
        eststo : qui reg female_w i.PostOct92 $u_p1 if `b'==1, vce(cluster x)
        qui boottest 1.PostOct92, seed(1234) cluster(x) nograph
        estadd scalar p = r(p): est`i'
}      

esttab *, cells(b(fmt(3)) se(fmt(3) star par([ ]))) noobs keep(1.PostOct92) stats(p)

Res.:

. esttab *, cells(b(fmt(3)) se(fmt(3) star par([ ]))) noobs keep(1.PostOct92) stats(p)

----------------------------------------------------------------------------
                      (1)             (2)             (3)             (4)  
                 female_w        female_w        female_w        female_w  
                     b/se            b/se            b/se            b/se  
----------------------------------------------------------------------------
1.PostOct92        -0.079           0.414           0.371          -0.022  
                  [0.070]         [0.477]         [0.369]         [0.410]  
----------------------------------------------------------------------------
p                   0.301           0.482           0.403           0.961  
----------------------------------------------------------------------------

[Sofia Hernandez]

Thank you so much, it works now! This was my first post on statalist so thank you both for the patience.