How to run a mixed ANOVA with Baseline values and other confounding variables as covariates

[Lisa Lim]

We conducted a randomised controlled trial on the effectiveness of music with and without special beats(i.e. Intervention and control groups) on pain ratings (dependent variable). The trial involved participants listening to a sound track for 30 min and chronic pain ratings were gathered before and after listening. I identified pre-treatment pain ratings as a potential covariate. Additionally, age, health and mental health ratings have been correlated with chronic pain, thus I would like to run them as covariates.

I am new to STATA and have managed to successfully run basic tests for descriptive statistics, Chi-squared tests and correlation tests etc. However, I have been getting more and more confused to run the ANOVA test even after searching and reading forums and online tutorials to run the tests. In addition, I have tried to compensate by doing independent T-test and paired T-test. However, these two tests do not allow me to account for covariates.

Details

• Groups: A = intervention (n = 28), B = control (n = 21) // Independent variable: Soundtrack_label
• Dependent variable: Pre_post_diff (i.e., The difference between the baseline and post-treatment ratings)(categorical variable)
• Covariate: Pain_pre (categorical variable)
• Other confounding variables as potential covariates: Age (continuous), Health_rating (categorical), MH_rating (categorical)

Analysis

• Mean change in Pre_post_diff, adjusted for Pain_pre, with a between-subjects factor (Soundtrack_label) and within-subjects factor (time - Pain_pre, Pain_post)
• Another test for the other covariates

Questions

• Have I labelled the within-subjects factor correctly? - i.e., Pain_pre, Pain_post
• How can I run a mixed ANOVA design? (Note: I have tried running ANOVA/ANCOVA with dependent variable as "Pre_post_diff", "Model" as Pain_pre and Soundtrack_labels specified as "Main effect", but I am not entirely sure I am running the correct thing)
• If I want to account for the other covariates, do I just continue adding them under the Model?

Do tell me if I should be providing more information as this is my first time posting. Thank you in advance for your assistance!

[Joseph Coveney]

. . .
• Dependent variable: Pre_post_diff (i.e., The difference between the baseline and post-treatment ratings)(categorical variable)
• Covariate: Pain_pre (categorical variable)
. . .

Literature exists about the use of a pre-post difference as your outcome measure, especially when you go ahead and use the pretreatment value as a covariate. You can start maybe here.

• How can I run a mixed ANOVA design?

You can use mixed, but it might be easier to at least start with a noniterative algorithm as in the following. (I don't know what you call your participants and so I'll make up a variable name.)

rename (Pain_pre Pain_post) pain#, addnumber(0)
preserve

/* Primary analysis. Examine the interaction term; the covariates will drop out. */
reshape long pain, i(participant_id) j(time)
xtreg pain i.Soundtrack_label##i.time c.(Age Health_rating MH_rating), i(participant_id) fe

// You can try this, too. The interaction term won't be identical but hopefully won't lead to divergent conclusion.
mixed pain i.Soundtrack_label##i.time c.(Age Health_rating MH_rating) || participant_id: , ///
    reml dfmethod(kroger) nolrtest nolog
contrast Soundtrack_label#time, small

/* Secondary analysis */
restore
regress pain1 i.Soundtrack_label c.(pain0 Age Health_rating MH_rating)

How coarse a categorical variable is the pain score? With such small sample size, you're kind of stuck with a linear model, but you'll want to take a look at the residuals anyway.

We conducted a randomised controlled trial . . .

Curious how you ended up with ns of 21 and 28. Block randomized 3:4 or was there a lot of differential dropout? And was randomization stratified on the baseline pain scores?

[Lisa Lim]

Hi Joseph, thank you so much for your responses, they are very helpful, below are my sub-responses to the individual parts.

1.
I read the article and realised that it would indeed be not advisable to run the tests based on pre-post differences considering that the effects may then be over/under-calculated. Thank you for the article.

2.
I have managed to run the codes as given, and seem to have finally gotten appropriate analyses, thank you! Except the " reml dfmethod(kroger) nolrtest nolog contrast Soundtrack_label#time, small " which I put after the mixed command. They stated that reml was not an existing command (r1999) and when I tried removing reml and run with dfmethod only, they indicated incorrect syntax (r198). Not really sure what went wrong there, perhaps my version does not have reml?

Additionally, I would like to clarify a few things. Firstly, for the xtreg test, it indicated that the 2.Soundtrack label, age, health rating and mental health rating are omitted because of collinearity which makes me a bit confused since they are not correlated and don't really have a linear relationship on a scatterplot. Thus, most of the results were omitted. Wondering whether I should refer more to the mixed ANOVA analysis which was the second one you suggested?

For the mixed multilevel analysis, everything was analysed, however, I would just like to confirm that I have interpreted the different labels correctly in the diagram below. I interpreted B as between subjects, 1.time as within subjects, and everything below Soundtrack_label#time as the interactions between the various covariates. I am very new to STATA, thus really appreciate your assistance for helping me to clarify my doubts.

-----

-----

Also, the pain score is from a numeric pain scale of 1 to 10, with 1 being no pain and 10 being the worst possible pain.

3.
The difference in ns was due to the way we did our randomisation, which was randomly generating 1s and 2s to indicate intervention/control. Since we did not hit our target sample size, the 1s and 2s are not equal, which unfortunately led to the present situation.

[Joseph Coveney]

. . . " reml dfmethod(kroger) nolrtest nolog contrast Soundtrack_label#time, small " which I put after the mixed command. They stated that reml was not an existing command (r1999) and when I tried removing reml and run with dfmethod only, they indicated incorrect syntax (r198). Not really sure what went wrong there, perhaps my version does not have reml?

You probably accidentally omitted the "continuation marker", which is ///. See the help file and scroll down to the Technical note section where it talks about it.

If you're still having trouble, then just append the reml . . . nolog line to the mixed . . . Participant_ID: line and run the code as one line:

mixed pain i.Soundtrack_label##i.time c.(Age Health_rating MH_rating) || Participant_ID, reml dfmethod(kroger) nolrtest nolog
contrast Soundtrack_label time Soundtract_label#time, small

Additionally, I would like to clarify a few things. Firstly, for the xtreg test, it indicated that the 2.Soundtrack label, age, health rating and mental health rating are omitted because of collinearity which makes me a bit confused since they are not correlated and don't really have a linear relationship on a scatterplot. Thus, most of the results were omitted.

Yeah, it is a little confusing. As a mater of fact, the interaction term that you get with xtreg . . ., fe is exactly the same that you got with your independent t-test on your Pre_post_diff variable as in

ttest Pre_post_diff, by(Soundtrack_label)

.If you had trouble with convergence with mixed, then that would still be available, because xtreg doesn't require iterative methods.

Wondering whether I should refer more to the mixed ANOVA analysis which was the second one you suggested?

Yes, you have no trouble with convergence (done in only two iterations) or with nonsensical results for variance components, so go ahead and use that second model.

For the mixed multilevel analysis, everything was analysed, however, I would just like to confirm that I have interpreted the different labels correctly in the diagram below. I interpreted B as between subjects, 1.time as within subjects, and everything below Soundtrack_label#time as the interactions between the various covariates.

Soundtrack_label is the between-subjects factor, and the B is the value label that you gave to the second group (which according to your description above is the control treatment group). time is the within-subjects factor; Soundtrack_label#time is their interaction, and it is the difference that you're looking for. Everything below are the covariates, but the model doesn't specify interactions of them and any other variable; they're just adjustments.

I neglected to mention it, but margins might be helpful to see the adjusted pain scores for each of the treatment groups at each of the observation time points. You would use the margins command after fitting the mixed-model ANOVA:

mixed pain i.Soundtrack_label##i.time c.(Age Health_rating MH_rating) || Participant_ID, ///
    reml dfmethod(kroger) nolrtest nolog
contrast Soundtrack_label time Soundtract_label#time, small
margins Soundtract_label#time

The adjusted means after margins will be labeled with the treatment groups (A and B) and time points (0 and 1), and will probably be easier to understand than the regression coefficients per se.

Also, the pain score is from a numeric pain scale of 1 to 10, with 1 being no pain and 10 being the worst possible pain.

OK.

The difference in ns was due to the way we did our randomisation, which was randomly generating 1s and 2s to indicate intervention/control. Since we did not hit our target sample size, the 1s and 2s are not equal, which unfortunately led to the present situation.

Understood.

[Lisa Lim]

Thank you Joseph for your comprehensive explanations and assistance. They have really helped me to figure out the statistical analyses that I was confused about. Wishing you a good week ahead!