Multiple test corrections

[paulvonhippel]

Hi all. I'm curious what Stata commands exist today for correcting p values for multiple tests. The ones I know about are mhtexp, mhtreg, wyoung, and rwolf2 -- all user-written and all reviewed in this 2021 blog post by David McKenzie at the World Bank.

I'm looking for commands that are general-purpose -- not limited to a specific context, such as ANOVA. Extra points if they also correct confidence intervals.

Best,
Paul

[Martyn Sherriff]

multproc in package st0035_1 from http://www.stata-journal.com/software/sj10-4 ?

Martyn

[paulvonhippel]

Hm, the link didn't work for me. I think you mean Frequentist q-values for multiple-test procedures?

[Martyn Sherriff]

Sorry.

search multproc

SJ-10-4 st0035_1 . . . . . . . . . . . . . . . Software update for smileplot
(help smileplot if installed) . . . R. Newson & The ALSPAC Study Team
Q4/10 SJ 10(4):691
updated to Stata 10 and addplot() option added

SJ-3-2 st0035 . . . . . . . . . . . Multiple-test procedures and smile plots
(help smileplot if installed) . . . R. Newson & The ALSPAC Study Team
Q2/03 SJ 3(2):109--132
implements multiple test procedures to control either
the family-wise error rate or the false discovery rate;
graphical tool for summarizing a set of multiple analyses

[paulvonhippel]

Thanks! I'm not sure I understand the difference between smileplot and qqvalues, both authored by Roger Newson . Is one supposed to replace the other?

[paulvonhippel]

To answer my own question: it seems smileplot compares uncorrected p values to a corrected alpha level, while qqvalues compares corrected p values (i.e., q values) to an uncorrected alpha level.
The two approaches are equivalent in that they both reject the same null hypotheses.

[Bruce Weaver]

For the record, both of the packages mentioned in #6 are available via SSC. And the second one is qqvalue (no s).

ssc describe smileplot
ssc describe qqvalue