boottest - seed option choice

[Aimee Kingsada]

Hi everyone,
I estimate a regression with instrumental variable but with few clusters, so I try to use the wild cluster bootstrap via boottest.
I have very different results when I change the random number seed : is it important to set seed? A way to choose this number?
Many thanks for those who read and for your answers.
Aimée

ivreg2 LNnb_totact (sect1=Z) V1 V2, cluster(timean)
boottest sect1, ptype(equaltail)
Wild bootstrap-t, null imposed, 999 replications, Wald test, bootstrap clustering by timean, Rademacher weights:
  sect1

                               z =     3.6677
    2 * min(Prob>|z|, Prob<-|z|) =     0.0260

95% confidence set for null hypothesis expression: [.08005, .8114]
boottest sect1, ptype(equaltail) seed(987654321)
Wild bootstrap-t, null imposed, 999 replications, Wald test, bootstrap clustering by timean, Rademacher weights:
  sect1

                               z =     3.6677
    2 * min(Prob>|z|, Prob<-|z|) =     0.0501
boottest sect1, ptype(equaltail) seed(12345)
	
	Wild bootstrap-t, null imposed, 999 replications, Wald test, bootstrap clustering by timean, Rademacher weights:
	  sect1
	
	                               z =     3.6677
	    2 * min(Prob>|z|, Prob<-|z|) =     0.0300
	
	95% confidence set for null hypothesis expression: [.09214, .7614]

[David Roodman]

Do the graphs look similar? They should help you understand what is going on. If the confidence curve is wiggling around 0.05 over a wide range, or only goes below 0.05 in a few places, then the results can be sensitive to the seed. Increasing the number of replications should make it more stable. You might also want to change the p threshold from 0.05 using the level() option.

[Tariku Getaneh]

Hi David,
Thank you for your contribution. I want to ask you a practical question regarding boottest command. I am using DiD approach on the pooled household surveys collected in different years.
Is it right to use survey weight in the following way and is there an issue with the bootttest command ?
eg. reg y treated_post treated post $Xlist [iw=weight]
boottest treated_post, cluster(region) reps(999) seed(293867483) madjust(bonferroni) nograph boottype(wild) weight(rademacher)

Vs

Here I added cluster option in my reg command, but the number of observations reduces compared to the previous command
eg. reg y treated_post treated post $Xlist [iw=weight], cluster(region)
boottest treated_post, cluster(region) reps(999) seed(293867483) madjust(bonferroni) nograph boottype(wild) weight(rademacher)

Which one is the correct way to run boottest? if both are wrong could you please show forward your suggestion?
Are there criteria for implementing the different weights? e.g. webb, normal, rademacher

Thanks in advance.

[David Roodman]

I realize it's a bit confusing, but the weights referred to in the boottest weight option are completely unrelated to the weights in the regress command.

Whether you should use weights in the regress command line is a something that people often get confused about. See What Are We Weighting For?. If you do use observation weights, you should do [pw=weight] not [iw=weight]. This will trigger the "robust" adjustment for heteroskedasticity.

The choice of auxiliary weights in the wild bootstrap is discussed in Fast and Wild. Usually the default, rademacher, is fine, or else webb.

[Tariku Getaneh]

Thank you so much, David. The reason why I use iw=weight is that I am using DHS cross-sectional data collected in different years. Also using (pw=weight) reduces the sample size if , cluster(region) option is added besides, F stat is also missing.
F(10, 10) = .
Prob > F = .

My question is it mandatory to use the cluster(region) option? since the number of regions is 11, SEs are incorrect anyway. Unfortunately, boottest doesn't offer corrected SE
But I could use svy: reg ..... boottest doesn't work thereafter. Is there a way around getting adjusted P values after accounting for a small number of cluster issues?