Different results between AMOS and Stata SEM, no upper bound for RMSEA in Stata

[Giorgos Chouliaras]

Hi

I have used Stata's SEM for a linear regression SEM model which worked fine and got some nice results.

Nevertheless I didn't get an upper bound for RMSEA:

RMSEA | 0.000
90% CI, lower bound | 0.000
upper bound | .
pclose | 0.994 Probability RMSEA <= 0.05

When I tested the exact same model (and same dataset) on AMOS (hoping to get a 90% ci) the results were different. All the regression estimates were identcal and SEs and p-values were almost the same with the exception of one estimate (effect of gender on a biochemical index) where, although the effect was the same, the SE and therefore the p-value differed significanlty (p=0.25 in Stata vs <0.001 in AMOS). In all other estimates the differences were negligible.

What was even more impressive (and frustrating) was the huge differences in RMSEA. Compared to the abovementioned values form Stata in AMOS the results were disappointing:
RMSEA

Model	RMSEA	LO 90	HI 90	PCLOSE
Default model	.359	.309	.411	.000

The differences are huge and what puzzles me is that I don't think it's some technical difference in estimation routines that has generated these results

Any thoughts why these discrepancies exist??

Thanks

[Dick Campbell]

It's hard to give you a sensible response without more information. Can you post the SEM code for the model you are estimating and show at least some of the output?

[wbuchanan]

Giorgos Chouliaras have you tried fitting the models with different starting values? And are both models using the same estimator? Typically when missing results are returned in Stata it is due to a problem with the model being fitted to the data (e.g., underidentification, etc...). Perhaps the problem is related to one program converging on a local maxima while the other did not?

[Giorgos Chouliaras]

Dear Prof Campbell thank you for you response

Below you can find the Stata and AMOS commands and output (Note: gender and diagnosis are binary variables, inter_age_gender is manually constructed interaction term for age and gender):

STATA

sem (gender -> height, ) (igf1 -> height, ) (hs_il6 -> igf1, ) (age -> igf1, ) (age -> height, ) (diagnosis-> hs_il6, ) (cort -> height, ) (inter_age_gender -> height, ), nocapslatent

------------------------------------------------------------------------------------
| OIM
| Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------------+----------------------------------------------------------------
Structural |
height <- |
igf1 | .0002572 .0001169 2.20 0.028 .0000281 .0004863
gender | -.1025718 .090797 -1.13 0.259 -.2805307 .0753871
age | .0327925 .0050489 6.49 0.000 .0228968 .0426882
cort | .004569 .001577 2.90 0.004 .0014783 .0076598
inter_age_gender | .0170243 .0081406 2.09 0.037 .001069 .0329796
_cons | .9326928 .0502661 18.56 0.000 .834173 1.031213
-----------------+----------------------------------------------------------------
igf1 <- |
hs_il6 | -9.357548 4.632129 -2.02 0.043 -18.43635 -.2787427
age | 11.86876 4.177524 2.84 0.004 3.680959 20.05655
_cons | 72.40768 49.1647 1.47 0.141 -23.95335 168.7687
-----------------+----------------------------------------------------------------
hs_il6 <- |
diagnosis | 2.436657 .7956872 3.06 0.002 .8771388 3.996175
_cons | 2.1918 .6347388 3.45 0.001 .9477348 3.435865
-------------------+----------------------------------------------------------------
var(e.height)| .0088677 .001691 .0061023 .0128863
var(e.igf1)| 11107.57 2118.13 7643.657 16141.23
var(e.hs_il6)| 8.057868 1.536575 5.545011 11.70949
------------------------------------------------------------------------------------
LR test of model vs. saturated: chi2(11) = 3.15, Prob > chi2 = 0.9886



estat gof, stats(all)

----------------------------------------------------------------------------
Fit statistic | Value Description
---------------------+------------------------------------------------------
Likelihood ratio |
chi2_ms(11) | 3.147 model vs. saturated
p > chi2 | 0.989
chi2_bs(18) | 102.105 baseline vs. saturated
p > chi2 | 0.000
---------------------+------------------------------------------------------
Population error |
RMSEA | 0.000 Root mean squared error of approximation
90% CI, lower bound | 0.000
upper bound | .
pclose | 0.994 Probability RMSEA <= 0.05
---------------------+------------------------------------------------------
Information criteria |
AIC | 1890.273 Akaike's information criterion
BIC | 1916.368 Bayesian information criterion
---------------------+------------------------------------------------------
Baseline comparison |
CFI | 1.000 Comparative fit index
TLI | 1.153 Tucker-Lewis index
---------------------+------------------------------------------------------
Size of residuals |
SRMR | 0.031 Standardized root mean squared residual
CD | 0.783 Coefficient of determination
----------------------------------------------------------------------------


AMOS

AMOS output

			Estimate	S.E.	C.R.	P	Label
hs_il6	<---	diagnosis	2.442	.805	3.033	.002
igf1	<---	hs_il6	-9.284	4.626	-2.007	.045
igf1	<---	age	11.866	4.181	2.838	.005
height	<---	igf1	.000	.000	2.189	.029
height	<---	age	.033	.004	8.277	***
height	<---	gender	-.103	.025	-4.052	***
height	<---	cort	.004	.002	2.986	.003
height	<---	inter_age_gender	.017	.002	7.897	***

RMSEA

Model	RMSEA	LO 90	HI 90	PCLOSE
Default model	.359	.309	.411	.000
Independence model	.337	.299	.376	.000

AIC

Model	AIC	BCC	BIC	CAIC
Default model	209.773	219.164
Saturated model	88.000	105.217
Independence model	276.404	279.535

ECVI

Model	ECVI	LO 90	HI 90	MECVI
Default model	3.814	3.144	4.620	3.985
Saturated model	1.600	1.600	1.600	1.913
Independence model	5.026	4.161	6.026	5.082

HOELTER

Model	HOELTER .05	HOELTER .01
Default model	11	13
Independence model	11	13

Execution time summary

Minimization:	.004
Miscellaneous:	.152
Bootstrap:	.000
Total:	.156

Giorgos

[Richard Williams]

Giorgos, as a sidelight, your Stata output would be much easier to read if you used code tags. See pt. 12 of the FAQ. As it is every space over one space gets deleted so nothing lines up correctly.

[Dick Campbell]

Aside from what Rich Williams just said regarding the unreadability of the output, I can't tell if AMOS and Stata are working with the same df. I haven't looked at AMOS output in years. Are the entries in the Hoelter table df or what? The way you have drawn the model you are explicitly assuming that diagnosis is uncorrelated with the other exogenous variables. But I can't tell if that drawing comes from AMOS or Stata. Stata would not make that assumption as a default. If the two models don't have the same df they are not equivalent.

[Giorgos Chouliaras]

Thank you all for your input

Prof Campbell, it's exactly what you mentioned about the covariances. It seems that Stata includes all covariances of exogenous variables in the calculations no matter if the user has included them or not in the model specification. On the contrary, AMOS requests verification of not including covariances, before running the model.

When all covariances are included in AMOS then the results and measures of GOF are identical

Grateful for your help

Regards

Giorgos