One-sample proportions test for clustered data

[Martine Nurek]

Hi there,

Thanks in advance for any help that you're able to provide.

I'm conducting a repeated-measures study (each participant saw several trials), where each trial had a binary outcome (1=success, 0=no success).
I observed 47 successes in 463 trials, and I'd like to know whether this proportion (10%) is significantly different from an hypothesised proportion (0%).

My research to date suggests that prtest with clustering would have been perfect, but it no longer allows for clustering.
Is anyone aware of a comparable test?

Sincere thanks again,
Martine

[Joro Kolev]

I have done these things long time ago in the context of my paper Kolev, G.I., Pina, G. and Todeschini, F., 2015. Decision making and underperformance in competitive environments: Evidence from the national hockey league. Kyklos, 68(1), pp.65-80.

My memory might be rusty, and somebody might have better advice, but as far as I remember:

1. the only (big) advantage of -prtest- is that it is exact, that is for any sample size it provides the exact p-value.

2. this exactness is lost when you have correlated measures as you have--an individual is samples multiple times. To my knowledge, there does not exist an exact test in this situation, for an exact test the observations have to be iid, and in repeated measures they are not, they are correlated.

From 1. and 2. my conclusion and advice to you is, just use linear regression on a constant

reg outcome, robust cluster(IndividualID)

and the t-test on the constant will give you the test agains the null of zero you desire.

[Martine Nurek]

That's very helpful Joro, thank you!
I'll take a look at your paper too.

[Joro Kolev]

That's very helpful Joro, thank you!
I'll take a look at your paper too.

Well, my paper is very educational, and also lots of fun to read if you like hockey, plus it is totally free as Middlesex paid for open access

But what is relevant to your analysis is on pages 70-71, the III. Setting up the test of rationality.

[Martine Nurek]

A huge thank you again!

And a follow-up question, if you don't mind:
The strategy that you've suggested works if the hypothesised proportion is 0%.
But what if the hypothesised proportion is something else (e.g., 12%)?
Are you aware of a technique that would work in this situation?

Thank you!

[Joro Kolev]

Thats super easy, you just use -test- after you fit the regression to test wether your constant is equal to particular value, like this:

. sysuse auto, clear
(1978 Automobile Data)

. keep if !missing(rep)
(5 observations deleted)

. reg foreign, robust cluster(rep)

Linear regression                               Number of obs     =         69
                                                F(0, 4)           =       0.00
                                                Prob > F          =          .
                                                R-squared         =     0.0000
                                                Root MSE          =      .4635

                                  (Std. Err. adjusted for 5 clusters in rep78)
------------------------------------------------------------------------------
             |               Robust
     foreign |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       _cons |   .3043478   .1522003     2.00   0.116     -.118228    .7269237
------------------------------------------------------------------------------

. test _cons=0.12

 ( 1)  _cons = .12

       F(  1,     4) =    1.47
            Prob > F =    0.2925