[Hadji Jalotjot]

Just some update. Good news and some bad.

Update 1
Good news: I managed to have convergence using Joseph's code.
Bad news: Not a good fit as RMSEA = 0.097


Update 2
Good news: I re-run Joseph's code using likerts items identified through EFA then CFA. EFA done per factor, not using all likert items all at the same time.
Bad news: Model did not converge

Update 3
Good news: I did a polychoric EFA using all Likert items at the same time. Quite successfull after encountering some errors due to big matrix. Only three factors came out.
Bad news. None yet and horpefully it stays thay way.

My next steps forward

1) I'll try to fit alternative models (similar to Update 1 and 2) and see if I can find a good fit.

IF NOT

2) I'll fit a model using the results of Update 3.


Before I forget. Thanks a lot guys.

[Hadji Jalotjot]

Eric has a point in that your particular sample doesn't really yield four factors in EFA, even of a polychoric correlation matrix (search polychoric) of all 22 variables. But for the following I'll assume that wherever you got the questionnaire from has done the necessary psychometric qualification of it for general use, and that your dataset's hard-pressed effort just reflects sampling variation. So, giving the benefit of the doubt to that source and taking the factor structure as a given, you can easily fit a correlated four-factor CFA to your dataset. See the first model in the code below—it converges cleanly in 10 iterations, and none of the coefficients (factor loadings, residual variances, latent factor variances and covariance) are out of line.

It's come up on the list before that fitting a second-order CFA with sem is more problematic than fitting an equivalent first-order CFA with an unstructured covariance matrix for the latent factors. But, from that fitted CFA, you can then take the fitted coefficients and use them as constraints for the second-order CFA, and then take those coefficients as starting values for an unconstrained version of the same second-order CFA. Newton-Raphson can be a little finicky when you're trying to get the starting values close enough. And in your case, the second-order CFA has a little less flexibility to fit the data well (three latent-factor-on-latent-factor factor loadings and the second-order latent factor's variance) than the first-order CFA (six covariance terms among the four first-order latent factors).

I've annotated the steps mentioned above in the code below. (For brevity and convenience, I've also shortened the names of the variables and latent factors.)

version 18.0

clear *

quietly import excel "s10 Likert responses.xlsx", sheet(Likert) firstrow

drop _index
rename (S10_101_? S10_102_? S10_103_? S10_104_?) (soc? ext? com? int?)

*
* Begin here
*
sem ///
(soc3 soc4 soc6 <- S) ///
(ext1 ext2 ext3 <- E) ///
(com1 com2 com3 com6 <- C) ///
(int1 int2 int4 int6 <- I), ///
nocnsreport nodescribe nolog
assert e(converged)
mata: rowsum(st_matrix("e(ilog)") :!= 0)

/* Obtaining relevant coefficients for use as constraints */
tempname B
matrix define `B' = e(b)["y1", "soc3:S".."/:var(I)"]
local cols : colfullnames `B'

// Exogenous latent factors will now be endogenous
foreach lf in S E C I {
local cols : subinstr local cols "/var(`lf')" "/var(e.`lf')"
}

// Creating constraints
local cns 0
foreach coef of local cols {
local ++cns
local b = `B'[1, `cns']
constraint define `cns' _b[`coef'] = `b'
}

// Constrained SEM in order to obtain coefficients as starting values
quietly sem ///
(soc3 soc4 soc6 <- S) ///
(ext1 ext2 ext3 <- E) ///
(com1 com2 com3 com6 <- C) ///
(int1 int2 int4 int6 <- I) ///
(S E C I <- A), constraints(1/`cns')
matrix define `B' = e(b)

// Unconstrained SEM with good starting values
sem ///
(soc3 soc4 soc6 <- S) ///
(ext1 ext2 ext3 <- E) ///
(com1 com2 com3 com6 <- C) ///
(int1 int2 int4 int6 <- I) ///
(S E C I <- A), ///
from(`B') nocnsreport nodescribe nolog
assert e(converged)
mata: rowsum(st_matrix("e(ilog)") :!= 0) // Converges in 10 iterations

exit

The model you're trying to fit (immediately above) converges in 10 iterations including the zeroeth one. I've attached the do-file and log file if you're interested in pursuing this approach further.

Is there any reason why you didn't use all of the variables of each scale?

Hi Joseph. I ran this code, got a convergent model but fit is not good.

I generated the modification indices. How to I put it in the code.

Also, can I run your code through the SEM Builder (visualization of the model)?

I alredy have the chart but I dont know if I can specify some of the commends in your code.

Thanks!

[Joseph Coveney]

I generated the modification indices. How to I put it in the code.

I don't know. In the way that I make use of SEMs, I've never needed to deal with modification indexes.

Also, can I run your code through the SEM Builder (visualization of the model)?

I don't know; I've never used the SEM Builder.

Based upon what you said in #8 above I thought that the questionnaire has already been validated, and so I've kind of been wondering ever since:what it is exactly that you're trying to do.

[Hadji Jalotjot]

I don't know. In the way that I make use of SEMs, I've never needed to deal with modification indexes.

I don't know; I've never used the SEM Builder.

Based upon what you said in #8 above I thought that the questionnaire has already been validated, and so I've kind of been wondering ever since:what it is exactly that you're trying to do.

1) Yes I adopted the questionnaire from a previous study with some modification to reflect the context of my study.
2) I manage ot include the suggested improvements (mostly covraiances) from the modification index but it did not converge.

[Hadji Jalotjot]

After countless attempts to find a model that will converge and with good fit...countless EFA CFA, then using different combinatons of Likert Items for each 1st order factors with literall no success (except using Joseph's code but poor fit of model), I think I finally got something going.

As suggested, I did a polychoric EFA on all likert items because it is now obvious that my priori latent factor models isnt fitting well. I got 3 latent factors which I then used with my 2nd order latent factor.

It managed to converge and the model has good fit. This is now my model using SEM builder...

This is just the begiining as I still have several variables to include...

Getting ahead of myself, if i use predict, can I use the predicted values in a regression analysis . I know it defeats the purpose of using SEM. Just in case I encounter similar problems as before, I plan to to that.

[Erik Ruzek]

You can predict the values of the latent variables and use those in subsequent analyses. It is not generally recommended because you lose the latent (uncertainty) aspect of the variable and turn it into something definitive and certain. You may as well create a mean of all the items in the scale - it will likely correlate with the predicted latent variable very highly (>0.9).

[Hadji Jalotjot]

In the end I predicted the factor scores for Ability and used it in path analysis.

I tried extending this model by adding more observed variables but the fear identification issues(and of course time) held me back.

My path model is quite complex (aka messy) so merging it with the Ability model will be quite a challeng. My sample size may not be enough to estimate all the free parameters.

I got results mostl using a -adf- option

[Richard Williams]

Sometimes when it is this hard to find results I wonder if the results are worth finding! Maybe you are revealing great hidden truths but other times you may be desperately struggling to find something that is trivial or that isn’t there at all. . I hope the former is true in your case. But if your model was correct and you had good data, I would think estimation would be fairly straightforward. I might consider using only single indicators or just adding items together and see if I could get a simpler model with just the essentials to work. Whatever you do, I hope it yields something spectacular!

[Hadji Jalotjot]

I am inclined to believe the results are worth finding hence difficult to find.

As I go along with my analysis and having a better understanding of SEM and my model, it is now becoming clearer.

Right, I fully understand the my two main issues - indentification and sample size. At first indeitification is the only problem I am aware of until I figured Stata a message under the results probably refer to my sample size, i.e. there just not enough data to get a solution/convergence.

Instead of using predicted score of Ability in path analysis, I am trying a lesser evil which is item parcelling as one of my 1st order factor has 15 items. That 15 items and my 266 sample size limits the number of observed variable I can includein my model.