Cluster reliability, latent variable measures and multilevel mixed-effects regression

[Bill Cecil]

This post describes a method for estimating individual cluster reliability following the multilevel mixed-effect prediction of a random intercept, slope or both, and their standard errors. Where the measure is a latent variable (random intercept or slope), reliability is the measure variance that is between-subject (y) compared to that which is within-subject (q), for clustered information that can be represented by a latent variable (random intercept or slope). In the case of a three level (two random intercepts) model representing hospital, surgeon and patient, there will be hospital clusters at the highest level and surgeon clusters at the second level. A single cluster representing 1 hospital (out of perhaps 20) may include 1200 cases and is represented by a single random intercept (ζ) with a standard error of SE1 (SE1^2 = the within subject variance or

q).

Each hospital (cluster) has its own random intercept; the variance of all the random intercepts is reported in the regression results as a random effect, the variance of the constant. In this example, the dependent variable is operative time or the duration of surgical procedures and a large random intercept represents longer duration surgeries while a small random intercept represents shorter duration surgeries.

The formula applied to estimate reliability for each cluster (hospital) is : r = y/(y+q)

which is equation 2.4 from Rabe-Hesketh and Skrondal, Multilevel Modeling, vol. 1.

An example where the standard error of the individual random intercept for hospital A =

0.0502928 and the variance of all the random intercepts for the 20 hospitals = 0.0058763;


Reliability =

0.0058763/(0.0058763 + (0.0502928^2) = 0.70

The reliability for all of the surgeon clusters can be represented graphically using a lowess curve:

I appreciate constructive comments.