unequal vs welch options for ttest: Why no mention of Welch (1938) in the documentation?

[Bruce Weaver]

The documentation for ttest says this about the unequal and welch options:

unequal specifies that the unpaired data not be assumed to have equal variances.

welch specifies that the approximate degrees of freedom for the test be obtained from Welch’s formula
(1947) rather than from Satterthwaite’s approximation formula (1946), which is the default when
unequal is specified. Specifying welch implies unequal.

As I (now) understand it, though, Welch (1938) proposed exactly the same adjustment to the df as Satterthwaite (1946). For that reason, the method unequal uses is often called the Welch-Satterthwaite method. If that is correct, I reckon Welch (1938) should be cited in the documentation, and the distinction between Welch (1938) and Welch (1947) should be made very clear.

Cheers,
Bruce

PS- Any members who are particularly interested in this issue may want to take a look at the ResearchGate discussion I launched the other day. This StackExchange thread is also relevant.

[Bruce Weaver]

Regarding the output from ttest with unequal and welch options, I suggest changes in labeling like the ones shown below.

. clear

. sysuse auto
(1978 automobile data)

. ttest weight, by(foreign) unequal

Two-sample t test with unequal variances
------------------------------------------------------------------------------
   Group |     Obs        Mean    Std. err.   Std. dev.   [95% conf. interval]
---------+--------------------------------------------------------------------
Domestic |      52    3317.115     96.4296    695.3637    3123.525    3510.706
 Foreign |      22    2315.909    92.31665    433.0035    2123.926    2507.892
---------+--------------------------------------------------------------------
Combined |      74    3019.459    90.34692    777.1936    2839.398    3199.521
---------+--------------------------------------------------------------------
    diff |            1001.206    133.4954                  734.32    1268.093
------------------------------------------------------------------------------
    diff = mean(Domestic) - mean(Foreign)                         t =   7.4999
H0: diff = 0                     Satterthwaite's degrees of freedom =  61.6199  <<< CURRENT
H0: diff = 0                 Welch-Satterthwaite degrees of freedom =  61.6199  <<< SUGGESTED

    Ha: diff < 0                 Ha: diff != 0                 Ha: diff > 0
 Pr(T < t) = 1.0000         Pr(|T| > |t|) = 0.0000          Pr(T > t) = 0.0000

. ttest weight, by(foreign) welch

Two-sample t test with unequal variances
------------------------------------------------------------------------------
   Group |     Obs        Mean    Std. err.   Std. dev.   [95% conf. interval]
---------+--------------------------------------------------------------------
Domestic |      52    3317.115     96.4296    695.3637    3123.525    3510.706
 Foreign |      22    2315.909    92.31665    433.0035    2123.926    2507.892
---------+--------------------------------------------------------------------
Combined |      74    3019.459    90.34692    777.1936    2839.398    3199.521
---------+--------------------------------------------------------------------
    diff |            1001.206    133.4954                734.5435    1267.869
------------------------------------------------------------------------------
    diff = mean(Domestic) - mean(Foreign)                         t =   7.4999
H0: diff = 0                             Welch's degrees of freedom =  64.3126  <<< CURRENT
H0: diff = 0                      Welch's (1947) degrees of freedom =  64.3126  <<< SUGGESTED

    Ha: diff < 0                 Ha: diff != 0                 Ha: diff > 0
 Pr(T < t) = 1.0000         Pr(|T| > |t|) = 0.0000          Pr(T > t) = 0.0000

[Bruce Weaver]

At the risk of being the only one who ever posts in this thread (), I'll add one more thing. After reading Thom Baguley's post in the RG discussion...

A bit of digging and Welch seems to have shown that the Welch (1947) isn't superior:

Aspin, Alice A., and B. L. Welch. “Tables for Use in Comparisons Whose Accuracy Involves Two Variances, Separately Estimated.” Biometrika 36, no. 3/4 (1949): 290–96. https://doi.org/10.2307/2332668. [see point 4 in the Appendix by Welch]

So it makes sense that the Welch-Satterthwaite correction is the standard.

...I wonder whether the welch option (i.e., the Welch 1947 adjustment) is even worth keeping. I know I'll not be using it. YMMV.