How can a CFA model estimated with SEM yield an SRMR>1?

[George Kephart]

I just estimated a simple CFA model with SEM, and obtained an SRMR (standardized root mean squared residual) of 2.212 . Unless I am mistaken, it should have a maximum value of 1.0. How is this possible?

The data is on a sample of 742 observations, and items are scored on a 10 pt scale

Command:
sem ( pih1-pih12 <- PIH), method(ADF)

The factor loadings look reasonable (no Heywood cases)

The model clearly does not fit well (not unexpected), and here are the fit indices:

-> estat gof, stats(all)

----------------------------------------------------------------------------
Fit statistic | Value Description
---------------------+------------------------------------------------------
Discrepancy |
chi2_ms(54) | 319.043 model vs. saturated
p > chi2 | 0.000
chi2_bs(66) | 748.898 baseline vs. saturated
p > chi2 | 0.000
---------------------+------------------------------------------------------
Population error |
RMSEA | 0.081 Root mean squared error of approximation
90% CI, lower bound | 0.073
upper bound | 0.090
pclose | 0.000 Probability RMSEA <= 0.05
---------------------+------------------------------------------------------
Baseline comparison |
CFI | 0.612 Comparative fit index
TLI | 0.526 Tucker-Lewis index
---------------------+------------------------------------------------------
Size of residuals |
SRMR | 2.212 Standardized root mean squared residual
CD | 0.926 Coefficient of determination
----------------------------------------------------------------------------

Of note,many of the standardized residuals could not be calculated, and thus are missing.

[Richard Williams]

On p. 148 of the Stata 14 sem manual it says

zerotolerance(tol) treats residuals within tol of 0 as if they were 0. tol must be a numeric value less than 1. The default is zerotolerance(0), meaning that no tolerance is applied. When standardized residuals cannot be calculated, it is because a variance calculated by the Hausman (1978) theorem turns negative. Applying a tolerance to the residuals turns some residuals into 0 and then division by the negative variance becomes irrelevant, and that may be enough to solve the calculation problem.

I can't say that I really understand that but it sounds worth trying.

[George Kephart]

Thanks Rich.

Adjusting that option does enable estimating more of the standardized residuals with "estat residuals, standardize", but this it is an option for the post estimation command and thus does not affect the estimation of SRMR. In the methods and formula section of the manual it is evident that the standardized residuals produced by estat are not the same as those used in the calculation of the SRMR.

I am making some progress in solving this issue. The problem lies in the use of the first moments in the calculation of SRMR, which according to the documentation some software packages ignore:
SRMR is calculated using the first and second moments unless sem option
nomeans was specified or implied, in which case SRMR is calculated based on second moments
only. Some software packages ignore the first moments even when available. See Hancock and
Mueller (2006, 157).

Estimating the model with the nomeans option yields an SRMR of .126. Looking at methods and formulas for SEM, this isolates the problem to the calculation of the standardize mean residuals and their incorporation into the calculation of SRMR as shown on p. 485 of the SEM manual.

So, I am wondering if there is a more general problem here with the computation of the SMSR in SEM, and where this leave me...

[George Kephart]

I am wondering how I can figure out why STATA is computing an SRMR greater than 1. Is there a way to access the code STATA uses?

I suppose I could also try writing my own program to compute the SRMR using the SEM results and the formula in the manual, but that is quite a bit of work (especially given that my STATA programming is rusty).

My more immediate concern is obtaining an accurate SRMR that I can report in my results. While the SRMRs for other models in my paper look reasonable, I no longer trust them! Any suggestions for a quick way to do this? Is there an equivalent package in R?

[George Kephart]

There is a confirmed bug in the Stata computation of the SRMR with SEM.

Rich and I have compared SRMRs from Stata with results from other software. MPlus does not report SRMRs when WLS is used for estimation, and Stata estimates of the SRMR are inconsistent with those produced by the the R Lavaan package, even where the Stata SRMR falls between 0 and 1.

I have been in touch with Stata technical support about this, provided them with all the information I have, and they have concluded that there is, indeed, a bug in SRMR computation. They plan to address this in an update.