paired t-test or Wilcoxon signed rank test?

[MJ Smith]

Hi, I am trying to decide whether a paired t-test or Wilcoxon signed rank test is better for my situation. I have pre-post data for a small number of individuals (n=9). There are two variables I am looking at to see if there was a change pre-post. One is a mean of proportions and another is a mean of durations. Below, I show histogram of the 4 variables involved. I conducted the Shapiro-Wilk W test for normal data, and in one variable (2nd one down in the histograms) the assumption of normality was violated (p=0.029), but it was not in the other 3.

I would prefer to use paired t-test rather than Wilcoxon signed rank test since differences in means would make more sense to my audience than differences in medians. I also read that the Wilcoxon signed rank requires a symmetrical distribution. However, I have a very small sample size. Any thoughts on which is the best test here?

thank you!

[Marcos Almeida]

With 9 observations, it may sound utopic to think about "normal distribution" (be it a test, be it a graph), more so with duration as a variable.

Let's just imagine the minimum sample size to perform a signed rank test, considering power = 80%, two-tailed alpha = 0.05 and a huge effect size (say, 0.7). We'd need 19 individuals for that matter, more than the double of the "real" sample".

In short, miracles apart, I believe permutation techniques (still keeping the frequentist hat) or Bayesian analysis would become an enticing alternative.

[MJ Smith]

Thank you very much for your reply! I do get strongly statistically significant results for both variables, both when using the paired t-test and the Wilcoxon. Does that influence your response?

[Nick Cox]

Why not show the data?

[MJ Smith]

Thank you for your reply. I wasn't sure if you meant show the raw data or results of running the tests, but here are the tests.

. ttest var1post = var1pre

Paired t test
------------------------------------------------------------------------------
Variable |     Obs        Mean    Std. Err.   Std. Dev.   [95% Conf. Interval]
---------+--------------------------------------------------------------------
var1post |       9    41.32322    1.266693     3.80008    38.40222    44.24422
 var1pre |       9    47.48633    1.292076    3.876229     44.5068    50.46587
---------+--------------------------------------------------------------------
    diff |       9   -6.163111    .9490831    2.847249     -8.3517   -3.974521
------------------------------------------------------------------------------
     mean(diff) = mean(var1post - var1pre)                        t =  -6.4938
 Ho: mean(diff) = 0                              degrees of freedom =        8

 Ha: mean(diff) < 0           Ha: mean(diff) != 0           Ha: mean(diff) > 0
 Pr(T < t) = 0.0001         Pr(|T| > |t|) = 0.0002          Pr(T > t) = 0.9999

.
. ttest var2post = var2pre

Paired t test
------------------------------------------------------------------------------
Variable |     Obs        Mean    Std. Err.   Std. Dev.   [95% Conf. Interval]
---------+--------------------------------------------------------------------
var2post |       9    65.03144    1.078618    3.235854    62.54415    67.51874
 var2pre |       9    50.11067    1.347224    4.041671    47.00396    53.21737
---------+--------------------------------------------------------------------
    diff |       9    14.92078    1.549006    4.647019    11.34876    18.49279
------------------------------------------------------------------------------
     mean(diff) = mean(var2post - var2pre)                        t =   9.6325
 Ho: mean(diff) = 0                              degrees of freedom =        8

 Ha: mean(diff) < 0           Ha: mean(diff) != 0           Ha: mean(diff) > 0
 Pr(T < t) = 1.0000         Pr(|T| > |t|) = 0.0000          Pr(T > t) = 0.0000

.
.
. signrank var1pre = var1post

Wilcoxon signed-rank test

        sign |      obs   sum ranks    expected
-------------+---------------------------------
    positive |        9          45        22.5
    negative |        0           0        22.5
        zero |        0           0           0
-------------+---------------------------------
         all |        9          45          45

unadjusted variance       71.25
adjustment for ties        0.00
adjustment for zeros       0.00
                     ----------
adjusted variance         71.25

Ho: var1pre = var1post
             z =   2.666
    Prob > |z| =   0.0077

.
. signrank var2pre = var2post

Wilcoxon signed-rank test

        sign |      obs   sum ranks    expected
-------------+---------------------------------
    positive |        0           0        22.5
    negative |        9          45        22.5
        zero |        0           0           0
-------------+---------------------------------
         all |        9          45          45

unadjusted variance       71.25
adjustment for ties        0.00
adjustment for zeros       0.00
                     ----------
adjusted variance         71.25

Ho: var2pre = var2post
             z =  -2.666
    Prob > |z| =   0.0077

[Nick Cox]

I mean the (raw or original) data, not results. (Your results suggest that you have little to worry about.)

[MJ Smith]

Thanks. Unfortunately, I'm not able to provide the raw data.

[Nick Cox]

If it were my data I would want to look at the distribution of the differences and worry about very moderate outliers if there were any.

[Marcos Almeida]

Amazingly enough, the SDs are relatively short as well as the CIs. Good news, and I wouldn't expect this. It seems all differences go to the same side (either all negative sum ranks, or all positive sum ranks). On account of this (amongst other aspects), the effect size (for alpha = 0.05 and power = 90%) was (a huge) 1.2, hence the statistical significance.

[MJ Smith]

Thanks! Below are histograms showing the distribution of the differences for the 2 variables. I did not notice any outliers from the data on the differences. I wonder if an approach here could be to present the paired t-test results, but also note for that the Wilcoxon test also gave consistent (statistically significant) findings?

[MJ Smith]

Just to add that I also conducted the Shapiro-Wilk W test for normal data on the 2 differences, and cannot reject the null hypothesis of normality (the p-values were 0.4. and 0.7). I realize this might not be that useful with small numbers.

[Nick Cox]

In your report -- unless you're obliged to suppress individual data points even there -- I would plot each post and pre variable versus rank on pre. Then the differences will be discernible as vertical distances. More at https://www.stata-journal.com/sjpdf....iclenum=gr0041