sem survey data - fit indices

[Dana Rose Garfin]

I am using STATA 13 to construct a fairly basic structural equation model with complex survey data (i.e., using pweights).

I have tried a variety of methods to get the software to provide gof estimates but thus far can only get STATA to provide SRMR and CD fit statistics with the notes: "model was fit with svy: prefix; only stats(residuals) valid" (if I use svyset) or "model was fit with vce(robust); only stats(residuals) valid" if I use the command [pweight=weight].

It looks like there is some controversy over whether it is appropriate to implement certain types of fit indices with sampling weights (Bollen, Tueller, & Oberski, 2013). Also the STATA manual indicates that some sem post-estimation procedures are not appropriate for survey estimation results. Is this why I am unable to obtain a larger number of common sem gof estimates?

Has anyone else encountered this problem? If so, how did you solve it?

Thank you!

[skolenik]

Dana,

per statalist rules, please provide full references (and on the forum, you can give a full link to the online version of the paper, so there are no excuses for not doing this).

I see no controversy here: no likelihood -- no fit indices. The issue of why there is no likelihood with survey data is a long-winded side conversation, but you can check Mary Thompson's "Theory of Sample Surveys" (Ch. 2 and 5) for explanations.

Stas

[Dana Rose Garfin]

Hi Stas,

Thank you very much for your reply (and the reference!) and I apologize for not initially posting the link to the Bollen reference (my first post here although I have benefited from the forum many times over the years).

The complete reference is:
Bollen, K. A., Tueller, S., & Oberski, D. (2013). Issues in the Structural Equation Modeling of Complex Survey Data. In Proceedings of the 59th World Statistics Congress. Hong Kong.

The link is http://www.statistics.gov.hk/wsc/STS010-P1-S.pdf

Thanks again!

Best,
Dana

[Amber Savage]

Hi Dana,

Were you ever able to solve your issue? I am having the same problem. Despite the apparent controversy I would still like the option to see fit statistics, even if only a chi-square test. If you found a work-around I would very much appreciate benefiting from your insights. If I find anything myself I will share it with you here.

Thanks!

Amber
(PhD student)

[Steve Samuels]

Just to note, correct spelling is "Stata", not "STATA", as, unlike, e.g. "SAS" and "SPSS", "Stata" is not an acronym.

[skolenik]

If you are absolutely desperate, you can construct fit indices yourself from e(ll) and other scalars that sem leaves behind. If I were asked to do this by a referee, though, I would have written an explanation as to why this is poor practice, and politely refuse to do this.

[Dana Rose Garfin]

According to the digging I have done, it is appropriate to use the Coefficient of Determination (which can be interpreted in the same way as an R square in linear regression) and the Standardized Root Mean Square Residual (less than .08 indicates "good" model fit) for complex survey data. This is what I ended up doing. Use the estat gof, stats(all) command after you run your model. I cited the Bollen et al (2013) conference proceeding that I posted the link to above as well as Hu & Bentler, 1999 (see link). We will see what the reviewers say!