Non-inferiority sample size calculation for proportions

[Tristan Lane]

Now this may be a massively stupid question however...

I am trying to sort out a sample size calculation for an RCT based on proportions.

Group 1 (control) is 0.68
Group 2 (experimental) is 0.80
Delta would be 0.05

When I use the power command I get wildly different (6x higher) sample size needed than if I were to use this online calculator:

http://powerandsamplesize.com/Calcul...or-Superiority

I think it is because I cannot work out where to put delta... or I’m doing it wrong (most likely). But I trust stata more than the online one...

Any help would be gratefully received.

[Steve Samuels]

Welcome to Statalist, Tristan! Be sure to read the FAQ, especially FAQ 12.

Non-inferiority and ordinary two-sample calculations are quite different. See, for example p. 832 of Fleiss et al. 2003. The contributed package art and command ssi (both at SSC) do non-inferiority calculations. I found them with

 search inferiority, all

Reference:
JL Fleiss, B Levin, and MC Paik (2003) Statistical Methods for Rates and Proportions, Third Edition, John Wiley & Sons, Hoboken NJ

[Tristan Lane]

Dear Steve,

Many thanks. I have already found the art and ssi and attempted to use as described. however the usage for proportions is difficult. Additionally there has been some suggestion on Statalist that these are relatively out of date compared to power.

I will put online my attempts once back from conference.

Thank you for your help.

tristan

[Steve Samuels]

power has no capabilities for non-inferiority calculations, so there's nothing to compare to art and ssi.

I've run the non-inferiority examples in the help of artbindlg and ssi with no problems. If you are having difficulties, then show us 1) the code you've tried (or, if you used an art dialog, the code that Stata wrote) and 2) the results that Stata produced. Put everything between CODE delimiters, described in FAQ 12.

[Tristan Lane]

Dear Steve,

Many thanks.

SSI Code:

ssi 0.80 0.05, alpha(0.05) power(0.9) noninferiority

ssi 0.80 0.05, alpha(0.05) power(0.9) noninferiority

Estimated sample size for two-sample comparison of proportions (non-inferiority)

Null hypothesis: p2 - p1 � delta (inferior), where:
pi (entered in command) is the overall proportion of participants
expected to experience the outcome if the treatments
are non-inferior, and delta is the smallest change in
proportions between groups (p2 - p1) which would still
be clinically important.

Alternative hypothesis: p2 - p1 < delta (non-inferior)

Note: a non-inferiority analysis is one-sided.

Assumptions:
power = 0.9000
alpha = 0.0500 (one-sided)
pi = 0.8000
delta = 0.0500
estimated required sample size (per group) = 1097

The total estimated sample size is 2194

ArtBin Code

 artbin, pr(0.80 0.68) ngroups(2) aratios(1 1) distant(0) alpha(0.05) power(0.9) onesided(0) ni(1)

ART - ANALYSIS OF RESOURCES FOR TRIALS (binary version 1.1.1 05aug2016)
------------------------------------------------------------------------------
A sample size program by Abdel Babiker, Patrick Royston & Friederike Barthel,
MRC Clinical Trials Unit at UCL, London WC1V 6LJ, UK.
------------------------------------------------------------------------------
Type of trial Non-inferiority - binary outcome
Statistical test assumed Comparison of 2 binomial proportions
P1 and P2.
Null hypothesis H0: P2-P1 = -0.120
Alternative hypothesis H1: P2-P1 = 0.000
Null variance estimation method Constrained maximum likelihood
Number of groups 2
Allocation ratio Equal group sizes

Anticipated event probabilities 0.800, 0.800

Alpha 0.050 (two-sided)
Power (designed) 0.900

Total sample size (calculated) 474
Expected total number of events 380
------------------------------------------------------------------------------

From SealedEnvelope - https://www.sealedenvelope.com/power...y-noninferior/ and same from http://powerandsamplesize.com/Calcul...or-Superiority

112 per group.
224 total.

This is why I'm confused! I'm obviously doing something wrong to get all the different results.
The smaller is better logistically but just want to get this write.

Tristan

[Joseph Coveney]

Noninferiority testing is tricky, and your confusion lies in what the null and alternative hypotheses are, and what the assumed true population rates of event are in the experimental and control treatments. This is why I prefer to use simulation. It forces you to think through the null and alternative hypotheses, forces you to explicitly implement your assumptions, and forces you to actually figure out how to test your null hypothesis using the same method that you intend to use once the data are gathered.

But for now, program the Blackwelder formula in Stata—save the following along your ado-path in an ado-file named as described:

*! blackwelder.ado
program define blackwelder, rclass
    version 15.1
    syntax , muc(real) [mue(real -1) delta(real 0.1) alpha(real 0.05) beta(real 0.80) noSpiel]

    local mue = cond(`mue' < 0, `muc', `mue')

    tempname f
    scalar define `f' = -invnormal(`alpha') + invnormal(`beta')
    scalar define `f' = `f' * `f'

    tempname numerator denominator
    scalar define `numerator' = `muc' * (1 - `muc') + `mue' * (1 - `mue')
    scalar define `denominator' = `muc' - `mue' - `delta'
    scalar define `denominator' = `denominator' * `denominator'

    return scalar n = ceil(`f' * `numerator' / `denominator')

    if "`spiel'" == "" spiel

    display in smcl as text _newline(1) "n (per group) = " return(n)
end

program define spiel
    version 15.1
    syntax

    display in smcl as text "Sample size required to reject null hypothesis of experimental worse (lower) than control by at least `delta'"
    display in smcl as text "against the alternative that experimental not worse than control by as much as `delta'"
    display in smcl as text "when the true rates in the populations are `muc' for control treatment and `mue' for experimental treatment"
    display in smcl as text "with `beta' power and `alpha' one-sided test size"
end

And then execute the following do-file. It will replicate all of the disparate results that you've shown in #5 for each of the two user-written Stata commands and for the online calculators.

version 15.1

* ssi's result
blackwelder , muc(0.8) mue(0.8) delta(0.05) alpha(0.05) beta(0.9) nospiel

* artbin's result
/* (Note: will be close, but won't match exactly because artbin uses
    different method for estimate of variance under the null hypothesis) */
blackwelder , muc(0.8) mue(0.8) delta(0.12) alpha(0.025) beta(0.9) nospiel
display in smcl as text "Total = " r(n) * 2

* Sealed Envelope's result
blackwelder , muc(0.68) mue(0.8) delta(0.05) alpha(0.05) beta(0.9) nospiel

exit

[Tristan Lane]

Thank you so much. You have really helped.

[Steve Samuels]

Very nice, Joseph!

Tristan Lane : If your Stata version number is less than 15.1, then substitute your number for "15.1" in the statements:

Version 15.1

You can find your version number with the command:

version

Thanks for using code delimiters. Although the FAQ doesn't say this (it should), Stata results also are much readable if you enclose them inside CODE blocks.

[Chris Dalton]

Joseph's program is very useful but when I leave out npspiel as in

blackwelder , muc(0.68) mue(0.8) delta(0.05) alpha(0.05) beta(0.9)

I just get blanks where the values of delta etc. should be in the "spiel". Have I missed something obvious here?

Regards

Chris

[Joseph Coveney]

Here, use the attached ado-file.


. blackwelder , muc(0.8) mue(0.8) delta(0.05) alpha(0.05) beta(0.9)
Sample size required to reject null hypothesis of experimental worse (lower) than control by at least .05
against the alternative that experimental not worse than control by as much as .05
when the true rates in the populations are .8 for control treatment and .8 for experimental treatment
with .9 power and .05 one-sided test size

n (per group) = 1097

.

[Chris Dalton]

Thanks Joseph.
I naively thought the "spiel" would automatically populate the parameters used in the blackwelder command.
Regards
Chris

[Rafat Noor]

Hello,
Can anyone help me with the code for power and sample size calculation for a cluster randomized non-inferior trial for two proportions?