[Hadji Jalotjot]

Reading David A. Kenny's blog on model identification, I now get that the Strutural Model part is unidentified.

Identification of the Structural Model

In the structural model, there is a set of structural equations. The causal variables are called exogenous variables and the effect variable is called the endogenous variable.
Unexplained variation is referred to as disturbance.

Rule A: Minimum Condition of Identifiability

Let k be the number of constructs in the structural model. Let q = k(k - 1)/2. The minimum condition of identifiability is that q must be greater than or equal to p where p equals the sum of:

a. number of paths,
b. number of correlations between exogenous variables that are not caused by any other variable,
c. number of correlations between the disturbance and an exogenous variables, and
d. number of correlations between disturbance.

In virtually all models, c is zero and in many models d is also zero. Theory places restrictions on a. Generally, b should be set at the maximum value; that is, all uncaused exogenous variables
should be correlated. If a structural model satisfies this condition, then the model may be identified. If it does not, the model is not identified; however, some but not all of the parameters
of the model may be identified.

Correct me if I am wrong. Looking at my visiual model
a. 14 less 5 that will be fixed to 1 = 9
b. 0 (Only Ability is the exogenous variable not caused by other variable)
c. 4 (I treated the 1st order latent variables as exogeous)
d. 0

q = (5*4)/2 = 10
p = 13

q<p --> Unidentified

So the next question is how do I address this?

[Joseph Coveney]

So I am running into another problem, then?

I would more inclined to wonder whether it is the problem and not another problem, but the means for all of the indicator variables for the Socialization latent factor are 4.3 or 4.4 (out of a maximum of 5?) and the error variance for the Socialization latent factor has collapsed to zero.

Have you tried fitting the four factors simultaneously with a diagonal covariance structure and then taking those coefficients as starting values for the final model as I suggested in #4 above? If so, how did it go?

Given the warning signs with the Socialization CFA submodel, you might want to also try fitting your second-order CFA first omitting Socialization and if that works then adding Socialization back in to fit the final model. Analogous to the previous suggestion, you'd carry forward the coefficient vector of the penultimate model (three first-order factors + the second-order factor) as starting values for the final model.

Those are about all that I could suggest given what you've shown so far. For anything further, at least from me, you'd have to attach your .dta file to a follow-up post or display it with the dataex command. If the latter, be sure to specify count(266)] as an option so that the entire dataset is provided.

[Hadji Jalotjot]

I would more inclined to wonder whether it is the problem and not another problem, but the means for all of the indicator variables for the Socialization latent factor are 4.3 or 4.4 (out of a maximum of 5?) and the error variance for the Socialization latent factor has collapsed to zero.

Have you tried fitting the four factors simultaneously with a diagonal covariance structure and then taking those coefficients as starting values for the final model as I suggested in #4 above? If so, how did it go?

Given the warning signs with the Socialization CFA submodel, you might want to also try fitting your second-order CFA first omitting Socialization and if that works then adding Socialization back in to fit the final model. Analogous to the previous suggestion, you'd carry forward the coefficient vector of the penultimate model (three first-order factors + the second-order factor) as starting values for the final model.

Those are about all that I could suggest given what you've shown so far. For anything further, at least from me, you'd have to attach your .dta file to a follow-up post or display it with the dataex command. If the latter, be sure to specify count(266)] as an option so that the entire dataset is provided.

I tried CFA for each latent factor (Socialization actually have a good fit)...then fitted the four factors at the same time (not good fit though, high RMSEA)...I'll fit it again using the coefficients from the ill-fitting model and see how it goes.

[Erik Ruzek]

I think having the data would allow us to be most helpful. There are weird things going on in the results, including the extremely low loadings of the Structural parameters and the negative residual variance on Socialization. What is the correlation of the 4 factors from the 4-factor CFA?

Personally, I wonder if there really are 4 factors. This is why I suggested you do an EFA. Just because you take a scale from the literature does not mean that it was analyzed carefully or correctly. Way too many people jump to CFA without looking at the EFA. The constraints imposed in CFA can make model fit look a lot better than it really is for the quality of the items, the tenability of the factors, etc.

[Hadji Jalotjot]

I think having the data would allow us to be most helpful. There are weird things going on in the results, including the extremely low loadings of the Structural parameters and the negative residual variance on Socialization. What is the correlation of the 4 factors from the 4-factor CFA?

Personally, I wonder if there really are 4 factors. This is why I suggested you do an EFA. Just because you take a scale from the literature does not mean that it was analyzed carefully or correctly. Way too many people jump to CFA without looking at the EFA. The constraints imposed in CFA can make model fit look a lot better than it really is for the quality of the items, the tenability of the factors, etc.

I am also starting to think the same way as you do Erik...This is an adopted set of Likert statements that are already categorized into latent factors but I might do an EFA risking dumping the theoretical model I used.

If anyone is interested, here is my data...

[Joseph Coveney]

Eric has a point in that your particular sample doesn't really yield four factors in EFA, even of a polychoric correlation matrix (search polychoric) of all 22 variables. But for the following I'll assume that wherever you got the questionnaire from has done the necessary psychometric qualification of it for general use, and that your dataset's hard-pressed effort just reflects sampling variation. So, giving the benefit of the doubt to that source and taking the factor structure as a given, you can easily fit a correlated four-factor CFA to your dataset. See the first model in the code below—it converges cleanly in 10 iterations, and none of the coefficients (factor loadings, residual variances, latent factor variances and covariance) are out of line.

It's come up on the list before that fitting a second-order CFA with sem is more problematic than fitting an equivalent first-order CFA with an unstructured covariance matrix for the latent factors. But, from that fitted CFA, you can then take the fitted coefficients and use them as constraints for the second-order CFA, and then take those coefficients as starting values for an unconstrained version of the same second-order CFA. Newton-Raphson can be a little finicky when you're trying to get the starting values close enough. And in your case, the second-order CFA has a little less flexibility to fit the data well (three latent-factor-on-latent-factor factor loadings and the second-order latent factor's variance) than the first-order CFA (six covariance terms among the four first-order latent factors).

I've annotated the steps mentioned above in the code below. (For brevity and convenience, I've also shortened the names of the variables and latent factors.)

version 18.0

clear *

quietly import excel "s10 Likert responses.xlsx", sheet(Likert) firstrow

drop _index
rename (S10_101_? S10_102_? S10_103_? S10_104_?) (soc? ext? com? int?)

*
* Begin here
*
sem ///
    (soc3 soc4 soc6 <- S) ///
    (ext1 ext2 ext3 <- E) ///
    (com1 com2 com3 com6 <- C) ///
    (int1 int2 int4 int6 <- I), ///
        nocnsreport nodescribe nolog
assert e(converged)
mata: rowsum(st_matrix("e(ilog)") :!= 0)

/* Obtaining relevant coefficients for use as constraints */
tempname B
matrix define `B' = e(b)["y1", "soc3:S".."/:var(I)"]
local cols : colfullnames `B'

// Exogenous latent factors will now be endogenous
foreach lf in S E C I {
    local cols : subinstr local cols "/var(`lf')" "/var(e.`lf')"
}

// Creating constraints
local cns 0
foreach coef of local cols {
    local ++cns
    local b = `B'[1, `cns']
    constraint define `cns' _b[`coef'] = `b'
}

// Constrained SEM in order to obtain coefficients as starting values
quietly sem ///
    (soc3 soc4 soc6 <- S) ///
    (ext1 ext2 ext3 <- E) ///
    (com1 com2 com3 com6 <- C) ///
    (int1 int2 int4 int6 <- I) ///
    (S E C I <- A), constraints(1/`cns')
matrix define `B' = e(b)

// Unconstrained SEM with good starting values
sem ///
    (soc3 soc4 soc6 <- S) ///
    (ext1 ext2 ext3 <- E) ///
    (com1 com2 com3 com6 <- C) ///
    (int1 int2 int4 int6 <- I) ///
    (S E C I <- A), ///
        from(`B') nocnsreport nodescribe nolog
assert e(converged)
mata: rowsum(st_matrix("e(ilog)") :!= 0) // Converges in 10 iterations

exit

The model you're trying to fit (immediately above) converges in 10 iterations including the zeroeth one. I've attached the do-file and log file if you're interested in pursuing this approach further.

Is there any reason why you didn't use all of the variables of each scale?

[Joseph Coveney]

Eric has a point . . .

Eric → Erik.

Apologies.

[Erik Ruzek]

No worries, Joseph. Nice work illustrating the subtleties of coaxing a second-order CFA from the data. I agree w/ Joseph that sampling variability has unfortunately hurt you in this case, Hadji. Maybe you could have wrung out the theoretical structure at a sample size of 450-500, The four factor model has unacceptably high correlations between factors (r between .80 and .99). Model fit of the second-order CFA (estat gof, stats(all)) is not up to the criteria I see cited in psychology and education (Cheung & Rensvold, 2002). The EFA suggests you have less than 4 factors - maybe 2. See code below. Note that you need to install polychoric and fapara from the web.

search polychoric
search fapara
polychoric soc3 soc4 soc6 ext1-ext3 com1-com3 com6 int1 int2 int4 int6
*Running using WLS from polychoric variance-covariance matrix
display r(sum_w) // 266
global N = r(sum_w)
matrix r = r(R)
factormat r, n($N)  // calling for an EFA on the var-cov matrix
rotate, blanks(.20) oblique oblimin(0) normalize factors(4)
fapara, reps(1000)

Clear kink at 3, suggesting 2 factors

[Richard Williams]

Richard, can you enlighten me when you say "The model as shown is not identified."..I've been reading on identification and in my mind this model is over-identified. This is my first time using SEM and any help is appreciated. Thanks!

A model can be both overidentified in some parts and underidentified in others. Ability was not identified. To fix its scaling, either its variance had to be fixed at 1 or one of the paths emanating from it had to be fixed at 1. The same was true for your first order factors -- all of them had 1 indicator path fixed at 1.

Based on your results, I wonder about your data. The loadings for the other three indicators of Ability are extremely small relative to the fixed path. The variance for Socialization is basically zero. Some of the coefficients reported do not have standard errors reported. Joe got the model to run fine with fake data, so I think there is some quirk with your real data that is causing you grief.

Are variables scaled wildly differently? Are some variables almost constant, e.g. 99% of the values are the same with only a few cases having different values? What are Social_* etc and how are they computed?

My guess is there is some problem with the data itself. Presenting the means and correlations for your observed data might help. Give a command something like

corr var1-var20, means

It could, of course, also be that it is a terrible horrible awful model. But check if there are fixable issues with the data first.

[Hadji Jalotjot]

A model can be both overidentified in some parts and underidentified in others. Ability was not identified. To fix its scaling, either its variance had to be fixed at 1 or one of the paths emanating from it had to be fixed at 1. The same was true for your first order factors -- all of them had 1 indicator path fixed at 1.

Based on your results, I wonder about your data. The loadings for the other three indicators of Ability are extremely small relative to the fixed path. The variance for Socialization is basically zero. Some of the coefficients reported do not have standard errors reported. Joe got the model to run fine with fake data, so I think there is some quirk with your real data that is causing you grief.

Are variables scaled wildly differently? Are some variables almost constant, e.g. 99% of the values are the same with only a few cases having different values? What are Social_* etc and how are they computed?

My guess is there is some problem with the data itself. Presenting the means and correlations for your observed data might help. Give a command something like

corr var1-var20, means

It could, of course, also be that it is a terrible horrible awful model. But check if there are fixable issues with the data first.

Thanls Richard.
I fixed the values of one path for each latent factor to 1 including one path from Ability...it still failed.
Im' beginning to think there is little variation in the responses of my Likert Scale questions...

I'll fitting a model using values from a non-convergent model as initial values as suggested in one of the comments. If that doesnt work, I'll run an EFA and CFA on the data...it may result on a totally new set of factors who knows.

here is the correlation matrix of the likert data I'm trying to fit a model with.

[Hadji Jalotjot]

Possibly. Try fitting the model stepwise, successively looking for problematic signs.

First, try fitting each of the component CFA models alone. Perhaps there is one that is such a poorly fitting model that it won't converge on its own and is bringing the joint model down.

If there's no problem at that step, then try fitting them all at once but not jointly, that is, remove the second-order latent factor and impose an independent covariance structure on the four first-order latent factors, using the covstructure(_LEx, diagonal) option.

If you get convergence there (you ought to if they all converged individually), then take the e(b) vector of coefficients as the set of starting values (assign the vector to a Stata matrix and feed it forward via the from() option) for the final model with the second-order latent factor added back in.

Hi Joseph. I just tried it now. Unfortunatelt the model still did not converge even after removing one 1st order latent factor that did have a good fit. Just to be sure I did it correct here is the code I used (after succesfully fitting each CFA model)

/// remove the second-order latent factor and impose an independent covariance structure on the four first-order latent factors, using the [covstructure(_LEx, diagonal) option --> did I do this correctly
///convergence at this model

sem (Internal -> Internal_2, ) (Internal -> Internal_4, ) (Internal -> Internal_5, ) (Internal -> Internal_6, ) (Combin -> Combination_1, ) (Combin -> Combination_2, ) (Combin -> Combination_3, ) (Combin -> Combination_4, ) (External -> External_1, ) (External -> External_2, ) (External -> External_3, ) (External -> External_4, ) (Social -> Social_1, ) (Social -> Social_2, ) (Social -> Social_4, ) (Social -> Social_5, ), covstruct(_lexogenous, diagonal) latent(Internal Combin External Social ) cov( Internal*External Combin*Internal Combin*External Social*Internal Social*Combin Social*External) nocapslatent

///getting the coefficients

matrix b = e(b)

///fitting with the 2nd order latent factor...I used the -skip- option because of the error in my first try

sem (Internal -> Internal_2, ) (Internal -> Internal_4, ) (Internal -> Internal_5, ) (Internal -> Internal_6, ) (Combin -> Combination_1, ) (Combin -> Combination_2, ) (Combin -> Combination_3, ) (Combin -> Combination_4, ) (Ability -> Internal, ) (Ability -> Combin, ) (Ability -> Social, ) (Social -> Social_1, ) (Social -> Social_2, ) (Social -> Social_4, ) (Social -> Social_5, ), from(b, skip) latent(Internal Combin Ability Social ) nocapslatent


///Message after fitting
///Warning: The LR test of model vs. saturated is not reported because the fitted model is not full rank. There appears to be 5 more fitted parameters than the data can support. convergence not achieved

Looks like it really is an identification problem at the structural model. More parameters to be estimated than my model

[Hadji Jalotjot]

Joseph has got you covered on how to diagnose problems. An additional issue to consider is the dimensionality of your items. Have you put all of them into an exploratory factor analysis? How many latent factors are retained?

You say you have Likert-scaled items but are using sem, which treats the items as continuous via an identity link. Have you tried running the model in gsem, using an ordinal link? There are lots of ways things can go wrong in measurement models.

Hi Erik. GSEM did not work even with 1st order latent factors only.

[Hadji Jalotjot]

Is there any reason why you didn't use all of the variables of each scale?

I did an EFA for each set of Likert items to choose those that have high loading in the factor. I used polychoric EFA for this. I got good results in the CFA except for the Externalization. I'll try EFA using all Likert Items...my initial results showed 3 factors only though. for some reason polychoric approach had an error when using all likert items.

[Hadji Jalotjot]

Eric has a point in that your particular sample doesn't really yield four factors in EFA, even of a polychoric correlation matrix (search polychoric) of all 22 variables. But for the following I'll assume that wherever you got the questionnaire from has done the necessary psychometric qualification of it for general use, and that your dataset's hard-pressed effort just reflects sampling variation. So, giving the benefit of the doubt to that source and taking the factor structure as a given, you can easily fit a correlated four-factor CFA to your dataset. See the first model in the code below—it converges cleanly in 10 iterations, and none of the coefficients (factor loadings, residual variances, latent factor variances and covariance) are out of line.

It's come up on the list before that fitting a second-order CFA with sem is more problematic than fitting an equivalent first-order CFA with an unstructured covariance matrix for the latent factors. But, from that fitted CFA, you can then take the fitted coefficients and use them as constraints for the second-order CFA, and then take those coefficients as starting values for an unconstrained version of the same second-order CFA. Newton-Raphson can be a little finicky when you're trying to get the starting values close enough. And in your case, the second-order CFA has a little less flexibility to fit the data well (three latent-factor-on-latent-factor factor loadings and the second-order latent factor's variance) than the first-order CFA (six covariance terms among the four first-order latent factors).

I've annotated the steps mentioned above in the code below. (For brevity and convenience, I've also shortened the names of the variables and latent factors.)

version 18.0

clear *

quietly import excel "s10 Likert responses.xlsx", sheet(Likert) firstrow

drop _index
rename (S10_101_? S10_102_? S10_103_? S10_104_?) (soc? ext? com? int?)

*
* Begin here
*
sem ///
(soc3 soc4 soc6 <- S) ///
(ext1 ext2 ext3 <- E) ///
(com1 com2 com3 com6 <- C) ///
(int1 int2 int4 int6 <- I), ///
nocnsreport nodescribe nolog
assert e(converged)
mata: rowsum(st_matrix("e(ilog)") :!= 0)

/* Obtaining relevant coefficients for use as constraints */
tempname B
matrix define `B' = e(b)["y1", "soc3:S".."/:var(I)"]
local cols : colfullnames `B'

// Exogenous latent factors will now be endogenous
foreach lf in S E C I {
local cols : subinstr local cols "/var(`lf')" "/var(e.`lf')"
}

// Creating constraints
local cns 0
foreach coef of local cols {
local ++cns
local b = `B'[1, `cns']
constraint define `cns' _b[`coef'] = `b'
}

// Constrained SEM in order to obtain coefficients as starting values
quietly sem ///
(soc3 soc4 soc6 <- S) ///
(ext1 ext2 ext3 <- E) ///
(com1 com2 com3 com6 <- C) ///
(int1 int2 int4 int6 <- I) ///
(S E C I <- A), constraints(1/`cns')
matrix define `B' = e(b)

// Unconstrained SEM with good starting values
sem ///
(soc3 soc4 soc6 <- S) ///
(ext1 ext2 ext3 <- E) ///
(com1 com2 com3 com6 <- C) ///
(int1 int2 int4 int6 <- I) ///
(S E C I <- A), ///
from(`B') nocnsreport nodescribe nolog
assert e(converged)
mata: rowsum(st_matrix("e(ilog)") :!= 0) // Converges in 10 iterations

exit

The model you're trying to fit (immediately above) converges in 10 iterations including the zeroeth one. I've attached the do-file and log file if you're interested in pursuing this approach further.

Is there any reason why you didn't use all of the variables of each scale?

Thanks again Joseph. You've been very helpful and informative as well. Even I have not found full success in the 2nd order latent factor (YET), I learned a lot from your suggestions even the basic stata commands that makes life easier.

I'll try my best to run this code, how daunting it may look to me right now.

[Hadji Jalotjot]

No worries, Joseph. Nice work illustrating the subtleties of coaxing a second-order CFA from the data. I agree w/ Joseph that sampling variability has unfortunately hurt you in this case, Hadji. Maybe you could have wrung out the theoretical structure at a sample size of 450-500, The four factor model has unacceptably high correlations between factors (r between .80 and .99). Model fit of the second-order CFA (estat gof, stats(all)) is not up to the criteria I see cited in psychology and education (Cheung & Rensvold, 2002). The EFA suggests you have less than 4 factors - maybe 2. See code below. Note that you need to install polychoric and fapara from the web.

search polychoric
search fapara
polychoric soc3 soc4 soc6 ext1-ext3 com1-com3 com6 int1 int2 int4 int6
*Running using WLS from polychoric variance-covariance matrix
display r(sum_w) // 266
global N = r(sum_w)
matrix r = r(R)
factormat r, n($N) // calling for an EFA on the var-cov matrix
rotate, blanks(.20) oblique oblimin(0) normalize factors(4)
fapara, reps(1000)

Clear kink at 3, suggesting 2 factors

I have not been successful running an EFA using all likert items. There is an error. I'll take a look at your code. Thanks!