ANCOVA vs Multilevel mixed-effects linear regression

[Jennifer May]

Hi all,

I'm a bit confused about the difference between an analysis of covariance and a multilevel mixed-effects linear regression.

I have a study with 2 arms (control and intervention), I am looking at the effect of therapy on a continuous outcome 'score'. I need to conduct a 2-level hierarchical model to account for participants clustered within therapists. I also will need to adjust for baseline 'score' and my stratification variables 'sex' and 'diagnosis'

In the protocol it says we are using analysis of covariance, but I have also been told I should be using mixed-effect models.

Does the -mixed- command do the same thing as an ANCOVA? Or are they two completely different things?

Any help greatly appreciated, thanks in advance.

[Clyde Schechter]

Does the -mixed- command do the same thing as an ANCOVA? Or are they two completely different things?

Neither. They do two different but closely related things.

Before delving into that, let me digress and point out that if you go for a mixed-effect model, it will have three levels, not just 2. That's because the mixed-model will require you to have multiple observations nested within patient (one for baseline and another, possibly several, depending on your design, for post-intervention). And the patients are then nested within therapists. So that's three levels.

There are two competing models for this set up. The ANCOVA model has only post-intervention observation(s) for each patient and includes the baseline outcome measurement as a covariate. The pure mixed-effects model has both pre- and post-intervention observations for each patient and does not include the baseline outcome measurement as a covariate. Algebraically, these models are equivalent:* the estimates from either can be simply transformed into the results from the other using linear algebra. In that sense they do the same thing. But the results do not look the same on the surface. That's because in the ANCOVA model the intervention effect, being adjusted for the baseline value as a covariate, is scaled down by a factor equal to the intraclass correlation of the outcome variable. In the mixed-effects model, lacking the covariate representation of the baseline (but including the baseline information as a separate observation in the data set) the intervention effect is estimated in its natural metric, not scaled down. For that reason, my personal preference is usually to use the mixed-effects model approach. I think the ANCOVA approach, however, has a longer history and investigators who grew up before mixed model estimation came into its own may be more comfortable with it.

Notwithstanding my personal preference for mixed models, there is one circumstance where the ANCOVA approach is required. If the baseline measurement is not made in the same way as the post-intervention measurements, then the error term for the baseline observation would not be exchangeable with the other observations, so that the assumptions for the mixed-effects model would be violated. This situation arises, for example, if the baseline measure is ascertained from medical records, whereas the post-intervention measures are ascertained by, say, a reference laboratory. Or if the baseline is obtained by self report but the post-intervention measures are the results of psychometric scales or expert evaluation. And so on. But if all of the outcome measures are obtained in the same way, you can use the pure mixed-effects model approach.

I should add that in your situation, even with the ANCOVA approach you will still need a mixed-effects model to deal with the nesting of patients within therapists. That would be a "hybrid" approach. It would be a mixed-effects analysis applied to the post-intervention observations only, and it would include the baseline value as a covariate.

*Statistically they are not quite equivalent: degrees of freedom are different, and this, in turn, affects standard errors, confidence intervals and test statistics in ways that are not simple algebraic transforms.

[Jennifer May]

Great thank you that is very helpful.