Goodness of fit indices unavailable when using SEM with vce(robust), what's the reason?

[Patrick Abi Nader]

Hello All,

I was able to find several posts asking why we can't get goodness of fit indices (RMSEA, CFI, TLI...) when use vce(robust) in SEM models. However, I haven't been able to find conclusive information on this topic and I am wondering if there any recent developments on this or a work around to get those fit indices. I have to use vce(robust) because my variables are not normally distributed, but I also need the fit indices to justify why I am using certain models.

Here are some posts that I found on the topic:

https://www.stata.com/statalist/arch.../msg00800.html
https://www.statalist.org/forums/for...of-fit-indices
https://www.statalist.org/forums/for...uster-clustvar

Thank you for your time and assistance with this,

Patrick

[Richard Williams]

xtdpdml is a shell for sem, i.e. it generates sem code. This handout discusses ways of dealing with non-normality, and what the pros and cons of each are. Perhaps vce(sbentler) would be a good choice for you.

https://www3.nd.edu/~rwilliam/dynami...%20xtdpdml.pdf

[Patrick Abi Nader]

Hello Richard,

I really appreciate your quick response. I just finished reading the document that you shared. The issue with using vce(sbentler) is that I can't use method(mlmv), and I can't get modification indices.

However, I am wondering if the following might be a strategy that I can use:

1) run my models with fiml, get goodness of fit and all modification indices.
2) once I have a good fitting model, use fiml and vce(robust) and compare estimations

Is that an approach that reviewers won't frown upon?

On another note, do you know if it is possible to conduct longitudinal measurement invariance in stata? I've only found information on group invariance.

Best wishes,
Patrick

[Richard Williams]

When there is no known perfect way to do things, a well-reasoned improvisation may fly (at least with me; I don't know about the people who review your papers).

It also helps if you can show that different approaches lead to more or less the same conclusions. Of course, if the conclusions are very different you have a problem.

I would do more checking to see if there isn't a better solution out there. In particular, I would check out what mplus can do, For example, when I ran these commands,

use https://www3.nd.edu/~rwilliam/statafiles/wages, clear
xtdpdml wks L.lwage, inv(ed) pre(L.union) ti(Baseline Model) vce(robust) gof fiml mplus(vcetry, r)

the GOF info I got was

----------------------------------------------------------------------------
Fit statistic        |      Value   Description
---------------------+------------------------------------------------------
Size of residuals    |
                SRMR |      0.019   Standardized root mean squared residual
                  CD |      0.313   Coefficient of determination
----------------------------------------------------------------------------
Note: model was fit with vce(robust); only stats(residuals) valid.

But, when I ran the same model in mplus, I got

RMSEA (Root Mean Square Error Of Approximation)

          Estimate                           0.000
          90 Percent C.I.                    0.000  0.021
          Probability RMSEA <= .05           1.000

CFI/TLI

          CFI                                1.000
          TLI                                1.012

Chi-Square Test of Model Fit for the Baseline Model

          Value                            594.959
          Degrees of Freedom                    99
          P-Value                           0.0000

SRMR (Standardized Root Mean Square Residual)

          Value                              0.023

Of course, that might be because mplus is doing something wrong, or that these numbers aren't very sensible.

As far as longitudinal measurement invariance, sem can impose all sorts of constraints. It may not do excactly what you want, but the xtdpml program imposes longitudinal constraints. You might want to check out the support page at

https://www3.nd.edu/~rwilliam/dynamic/index.html

An example of code it can generate is

use https://www3.nd.edu/~rwilliam/statafiles/wages, clear
keep wks lwage union ed id t
xtset id t
reshape wide wks lwage union, i(id) j(t)
sem (wks2 <- wks1@b1 lwage1@b2 union1@b3 ed@b4 Alpha@1 E2@1) ///
(wks3 <- wks2@b1 lwage2@b2 union2@b3 ed@b4 Alpha@1 E3@1) ///
(wks4 <- wks3@b1 lwage3@b2 union3@b3 ed@b4 Alpha@1 E4@1) ///
(wks5 <- wks4@b1 lwage4@b2 union4@b3 ed@b4 Alpha@1 E5@1) ///
(wks6 <- wks5@b1 lwage5@b2 union5@b3 ed@b4 Alpha@1 E6@1) ///
(wks7 <- wks6@b1 lwage6@b2 union6@b3 ed@b4 Alpha@1), ///
var(e.wks2@0 e.wks3@0 e.wks4@0 e.wks5@0 e.wks6@0) var(Alpha) ///
cov(Alpha*(ed)@0) cov(Alpha*(E2 E3 E4 E5 E6)@0) ///
cov(_OEx*(E2 E3 E4 E5 E6)@0) cov(E2*(E3 E4 E5 E6)@0) ///
cov(E3*(E4 E5 E6)@0) cov(E4*(E5 E6)@0) cov(E5*(E6)@0) ///
cov(union3*(E2)) cov(union4*(E2 E3)) cov(union5*(E2 E3 E4)) ///
cov(union6*(E2 E3 E4 E5)) ///
iterate(250) technique(nr 25 bhhh 25) noxconditional

All those @ are for parameters constrained to be equal.

[Patrick Abi Nader]

Your response was very helpful Richard. Thank you
Patrick