Hello
I would be most grateful if you could help with a problem I am experiencing using Stata.
Essentially I am obtaining substantially different results when using mixed effects model as compared with repeated measures ANOVA and I am not sure why this is happening.
Experiment:
- Two groups of subjects (healthy subjects – HC; diseased subjects – DS) performed an experimental task.
- There is one dependent variable “DV”.
- Each subject performed four different experimental conditions, defined by a combination of 2 independent variables:
I am interested in measuring the effect of “group”, “difficulty” and “distraction” along with their respective interactions on the dependent variable (DV).
I previously used mixed effects model ("mixed command") but in the field I am working reviewers prefer to see ANOVA instead of mixed effects model.
Problem:
groupXdistraction is significant when using mixed but now when using anova, and this makes a BIG difference in terms of result interpretation
do you think I am misusing ANOVA command? do you see any obvious reason for this to happen?
I am using the following commands:
Please see output below:
Many thanks
Jaime
I would be most grateful if you could help with a problem I am experiencing using Stata.
Essentially I am obtaining substantially different results when using mixed effects model as compared with repeated measures ANOVA and I am not sure why this is happening.
Experiment:
- Two groups of subjects (healthy subjects – HC; diseased subjects – DS) performed an experimental task.
- There is one dependent variable “DV”.
- Each subject performed four different experimental conditions, defined by a combination of 2 independent variables:
- Difficulty (easy 0, difficult 1)
- Distraction (without distraction 0, with distraction 1)
I am interested in measuring the effect of “group”, “difficulty” and “distraction” along with their respective interactions on the dependent variable (DV).
I previously used mixed effects model ("mixed command") but in the field I am working reviewers prefer to see ANOVA instead of mixed effects model.
Problem:
groupXdistraction is significant when using mixed but now when using anova, and this makes a BIG difference in terms of result interpretation
do you think I am misusing ANOVA command? do you see any obvious reason for this to happen?
I am using the following commands:
Code:
mixed dv i.group##i.difficult##i.distraction ||id2:, stddev covar(unstr)
Code:
anova dv i.group / id2|i.group i.difficult i.distraction i.group#i.difficult i.group#i.distraction i.difficult#i.distraction i.group#i.difficult#i.distraction, repeated(difficult distraction)
Please see output below:
Code:
mixed dv i.group i.difficult i.distraction i.group#i.difficult i.group#i.distraction i.difficult#i.distraction i.group#i.difficult#i
> .distraction ||id2:, stddev covar(unstr)
Note: single-variable random-effects specification in id2 equation; covariance structure set to identity
Performing EM optimization:
Performing gradient-based optimization:
Iteration 0: log likelihood = -707.02709
Iteration 1: log likelihood = -707.02709
Computing standard errors:
Mixed-effects ML regression Number of obs = 168
Group variable: id2 Number of groups = 42
Obs per group: min = 4
avg = 4.0
max = 4
Wald chi2(7) = 22.57
Log likelihood = -707.02709 Prob > chi2 = 0.0020
---------------------------------------------------------------------------------------------
dv | Coef. Std. Err. z P>|z| [95% Conf. Interval]
----------------------------+----------------------------------------------------------------
1.group | 2.708238 7.587663 0.36 0.721 -12.16331 17.57978
1.difficult | 1.608696 3.428798 0.47 0.639 -5.111626 8.329017
1.distraction | -3.217391 3.428798 -0.94 0.348 -9.937713 3.50293
|
group#difficult |
1 1 | 8.812357 5.097882 1.73 0.084 -1.179309 18.80402
|
group#distraction |
1 1 | 11.08581 5.097882 2.17 0.030 1.094147 21.07748
|
difficult#distraction |
1 1 | 7.173913 4.849053 1.48 0.139 -2.330057 16.67788
|
group#difficult#distraction |
1 1 1 | -12.12128 7.209494 -1.68 0.093 -26.25163 2.009068
|
_cons | 47.73913 5.103407 9.35 0.000 37.73664 57.74162
---------------------------------------------------------------------------------------------
------------------------------------------------------------------------------
Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval]
-----------------------------+------------------------------------------------
id2: Identity |
sd(_cons) | 21.53666 2.523021 17.11828 27.09547
-----------------------------+------------------------------------------------
sd(Residual) | 11.62762 .7324712 10.27709 13.15563
------------------------------------------------------------------------------
LR test vs. linear regression: chibar2(01) = 137.12 Prob >= chibar2 = 0.0000
.
end of do-file
. do "C:\Users\TIAGOT~1\AppData\Local\Temp\STD00000000.tmp"
. anova dv i.group / id2|i.group i.difficult i.distraction i.group#i.difficult i.group#i.distraction i.difficult#i.distraction i.group
> #i.difficult#i.distraction, repeated(difficult distraction)
Number of obs = 168 R-squared = 0.8412
Root MSE = 11.9148 Adj R-squared = 0.7790
Source | Partial SS df MS F Prob > F
----------------------+----------------------------------------------------
Model | 90247.9303 47 1920.16873 13.53 0.0000
|
group | 3857.21892 1 3857.21892 1.85 0.1819
id2|group | 83601.4894 40 2090.03724
----------------------+----------------------------------------------------
difficult | 1797.30807 1 1797.30807 12.66 0.0005
distraction | 345.720878 1 345.720878 2.44 0.1213
group#difficult | 78.7842568 1 78.7842568 0.55 0.4578
group#distraction | 262.744688 1 262.744688 1.85 0.1762
difficult#distraction | 12.8954043 1 12.8954043 0.09 0.7636
group#difficult# |
distraction | 382.181119 1 382.181119 2.69 0.1035
|
Residual | 17035.403 120 141.961692
----------------------+----------------------------------------------------
Total | 107283.333 167 642.41517
Between-subjects error term: id2|group
Levels: 42 (40 df)
Lowest b.s.e. variable: id2
Covariance pooled over: group (for repeated variables)
Repeated variable: difficult
Huynh-Feldt epsilon = 1.0256
*Huynh-Feldt epsilon reset to 1.0000
Greenhouse-Geisser epsilon = 1.0000
Box's conservative epsilon = 1.0000
------------ Prob > F ------------
Source | df F Regular H-F G-G Box
----------------------+----------------------------------------------------
difficult | 1 12.66 0.0005 0.0005 0.0005 0.0005
group#difficult | 1 0.55 0.4578 0.4578 0.4578 0.4578
Residual | 120
---------------------------------------------------------------------------
Repeated variable: distraction
Huynh-Feldt epsilon = 1.0256
*Huynh-Feldt epsilon reset to 1.0000
Greenhouse-Geisser epsilon = 1.0000
Box's conservative epsilon = 1.0000
------------ Prob > F ------------
Source | df F Regular H-F G-G Box
----------------------+----------------------------------------------------
distraction | 1 2.44 0.1213 0.1213 0.1213 0.1213
group#distraction | 1 1.85 0.1762 0.1762 0.1762 0.1762
Residual | 120
---------------------------------------------------------------------------
Repeated variables: difficult#distraction
Huynh-Feldt epsilon = 1.0256
*Huynh-Feldt epsilon reset to 1.0000
Greenhouse-Geisser epsilon = 1.0000
Box's conservative epsilon = 1.0000
------------ Prob > F ------------
Source | df F Regular H-F G-G Box
----------------------+----------------------------------------------------
difficult#distraction | 1 0.09 0.7636 0.7636 0.7636 0.7636
group#difficult# |
distraction | 1 2.69 0.1035 0.1035 0.1035 0.1035
Residual | 120
---------------------------------------------------------------------------
Many thanks
Jaime
Comment