Sample size estimation for comparison of three proportions: Control, Treatment1 and Treatment2

[Dora Pearce]

Dear StataList,
How can I calculate the sample size requirements for proportions in 3 (three) groups, namely: Control, Treatment1, and Treatment 2, when the outcome proportions typically range between 2% and 20%? Indeed, is it necessary to compare 3 groups - is it sufficient to compare Treatment1 vs Control and Treatment2 vs Control, and Treatment1 vs Treatment2, when I wish to establish that at least one of the treatment groups differs statistically from the Control group, and that Treatment1 differs significantly from Treatment2. Ideally, I would like to detect statistically significant differences between Controls and each of the treatment groups and between Treatment1 and Treatment2.
I will appreciate your expert comments.
Dora Pearce

[Phil Bromiley]

Stata 15 and 16 (and I think some earlier versions) have an entire manual for sample size and power calculations. I've never used it so I don't know if it does exactly what you want.

[Leonardo Guizzetti]

There are several things to consider for your question, so I'll tackle them in order of broader to narrower scope. For the rest of this post, I will assume that you are seeking advice on a randomized controlled trial.

Is it necessary to compare 3 groups? This depends entirely on the scientific question that you wish to answer. The setup you describe could be for a 3-arm trial with one control and two difference treatment groups (T1 and T2. for short). Or, T1 and T2 could be completely different treatments, such as two competing drugs. You mention that you would ideally like to compare each treatment against the control group, but this leaves open the question of whether there is interest in comparing the two treatments against each other, which in itself could be quite interesting. The number and nature of your comparisons will directly impact the sample size calculation, so you may wish to consult with a statistician for your specific scenario.

This is the starting approach I would consider, which is quite general and flexible. I would consider using logistic regression with an indicator variable for treatment group as the starting analysis framework, and then assess different scenarios and operating properties by simulation. For example, I would simulate binary outcomes for each group, given a hypothesized rate of success for each group and with the desired number of subjects in each group, then look at the operating properties of all group-wise contrasts I would be interested in.)

A second approach (which is also based on a logistic regression model) is based on the Cochran-Armitage trend test. This is documented with a handy sample size calculation tool in Stata by typing -help power trend-. However, this approach has the drawback that it is only valid for testing a linear trend in proportions, which could be sensible when looking for a dose-response effect. I can't say if this is right for your scenario.

[Dora Pearce]

Thank you, Leonardo, for your detailed response, and given the likely outcome proportions for the C, T1 and T2 groups, using Stata's sample size calculation for the "linear trend in proportions" test may be the best option for me.

Alternatively, would it be feasible to calculate the sample size required for a two proportion comparison, such that either T1 OR T2 differs from C (or T1 differs from T2) but using a Bonferroni adjusted significance level (alpha = 0.017 for 3 comparisons, say)?

And thank you, Phil, for your suggestion, I will check out the new Stata 16 Sample-size Manual.
Dora

[Joseph Coveney]

using a Bonferroni adjusted significance level (alpha = 0.017 for 3 comparisons, say)?

With only three comparisons, it might not be necessary to do any adjustments for multiple comparisons. See this post and the immediately following one (#7) in that thread for details.

[Dora Pearce]

Thank you, Joseph.
Being able to avoid making an adjustment for multiple comparisons would certainly help to minimize the required sample size per group.
Dora

[Hutchins Yeo]

There are several things to consider for your question, so I'll tackle them in order of broader to narrower scope. For the rest of this post, I will assume that you are seeking advice on a randomized controlled trial.

Is it necessary to compare 3 groups? This depends entirely on the scientific question that you wish to answer. The setup you describe could be for a 3-arm trial with one control and two difference treatment groups (T1 and T2. for short). Or, T1 and T2 could be completely different treatments, such as two competing drugs. You mention that you would ideally like to compare each treatment against the control group, but this leaves open the question of whether there is interest in comparing the two treatments against each other, which in itself could be quite interesting. The number and nature of your comparisons will directly impact the sample size calculation, so you may wish to consult with a statistician for your specific scenario.

This is the starting approach I would consider, which is quite general and flexible. I would consider using logistic regression with an indicator variable for treatment group as the starting analysis framework, and then assess different scenarios and operating properties by simulation. For example, I would simulate binary outcomes for each group, given a hypothesized rate of success for each group and with the desired number of subjects in each group, then look at the operating properties of all group-wise contrasts I would be interested in.)

A second approach (which is also based on a logistic regression model) is based on the Cochran-Armitage trend test. This is documented with a handy sample size calculation tool in Stata by typing -help power trend-. However, this approach has the drawback that it is only valid for testing a linear trend in proportions, which could be sensible when looking for a dose-response effect. I can't say if this is right for your scenario.

This is an old post, but it relates to something I've been struggling to answer. So you proposed a specific approach to deal with multiple-arm designs. When I Google it, there seems to be an infinite number of ways to address this. By contrast, there is a very simple way to address a standard 2-arm treatment design. Why is that the case? Is there no consensus on how to do power analysis for multiple-arms? Like why isn't there a G*Power or PowerUp or -power- equivalent? Why can't I just specify -power, n(3)- or something?