Plotting Lots of Coefficients

[Lindsey Buck]

Hi all,

I have a question about plotting some estimates from a bunch of regressions. My plan is to randomly assign a bunch of zip codes in my data to a control or treatment group and run a regression, then reassign them again, and do this 1,000 times as a placebo test to show that my actual result (when they are really assigned to their real control or treatment) is valid. I want to make some sort of plot of the estimates to show they should be centered around zero, right? Do you have thoughts on a program or way to do this? Thanks!
Lindsey

[Lindsey Buck]

A note from my advisor on this: "You need to draw from a random distribution (say N(0,1)); assigning some proportion of the control group "treated" status. Say if the draw goes above 0.5: ideally this should be the same number of "true" treated zip codes in the data, or at least the same proportion. Then estimate the effects and repeat 1000 times. Be sure to "set seed " and save the seed so you can replicate your results."

[FernandoRios]

I would say that this call for simulate

Example

webuse union3, clear
program tosim, eclass
    capture drop fake_union
    gen fake_union=runiform()< .21
    reg wage age grade smsa black tenure fake_union
end
simulate, reps(200):tosim
kdensity _b_fake_union

[Mike Lacy]

Fernando's approach will work fine, but I'd like to point to some built-in Stata capacity that seems to be unknown to people interested in what my economist friends call a "placebo" test. To my understanding, that's another term for what is commonly known as a "permutation" test, about which see -help permute-.

Stata's -permute- procedure implements this kind of procedure by repeatedly shuffling one variable across observations (i.e., sampling with replacement from the observed distribution of the permuted variable and assigning those values to the variable before doing an analysis). In the current context, the built-in functionality of -permute- obviates the need to create a program to do the resampling. Using Fernando's data example, one could do this:

webuse union3, clear
tempfile temp  // to hold the results from each repetition, if something beyond a p-value is desired
permute union b = _b[union], saving(`temp') reps(1000):reg wage age grade smsa black tenure union
use `temp', clear
kdensity b

[FernandoRios]

New day, something new to learn!
Thank you Mike, I had no idea about this command.
Best Wishes

[Rich Goldstein]

you don't say how many zip codes you have; while I generally agree with what Mike Lacy suggests, I do want to note a new option in Stata 17 for the -permute- command, namely the "enumerate" command; if you have a relatively small number, this option should be used as it then gives an exact result; if you have a larger number, then use without this option and get an approximate result based on randomly drawing permutations; what is considered large or small here varies by the person doing the work (how much do you want exact results), and their hardware as enumeration can take a long time; note that you can use the "comb()" function to find how many possible combinations there are here to help in making the decision

[Lindsey Buck]

Thank you so much to you all!!!!

[Lindsey Buck]

tempfile temp // to hold the results from each repetition
permute city_pit beta1 = _b[beta1], saving(`temp') reps(1000):reghdfe hous_all_act_ind beta1 gender2 race2 age_at_refer highrisk lowrisk if ym>=`=tm(2019m1)', absorb(i.hh_zip2 i.yearmonth)
use `temp', clear
kdensity beta1


Does anyone know why this didn't come out with any plot? So city_pit is my treatment variable (0 if not treated, 1 if treated) and beta1 is the variable interaction between prepost and city_pit. I have something wrong but I am not quite sure what.