Create Distribution of P-values Across Regressions

[Jordan Fisher]

I am regressing 100 repetitions with my randomly assigned treatment variable x1 as my variable of interest. How can I extract the relevant p-values from each of the 100 regressions and create a distribution of p-values as a robustness check? Does it have anything to do with bootstrapping?

forvalues i = 1(1)100{
capture drop x1
gen x1 = rbinomial(1, 0.5)
reg y x1 x2 x3 x4, r
test x1
}

Thank you!

[Rich Goldstein]

first, your loop never refers to `i' and thus you are not getting what you think you are getting

second, see

help statsby
help runby

for easier ways to get what you seem to want

if you want to stay with something like the procedure you have, the p-values are kept in the "r(table)" matrix after each estimation - so you could extract them and place them somewhere

[Leonardo Guizzetti]

Here is a more involved approach, but can be extended to be more flexible. The basic idea is the make a macro program that handles the simulation aspect and returns the p-value. Then you can pass this through Stata's -simulate- command to handle the repetition and collection of the results.

clear *
cls

set seed 123

cap program drop mysim
program define mysim, rclass
  version 15.1
  clear
    sysuse auto
    gen x = rbinomial(1, 0.5)
    reg turn length i.x
  test 1.x
    return scalar p = r(p)
end

simulate p=r(p), reps(50): mysim
hist p

[Jordan Fisher]

Thank you!