Latent class in Stata: missing values of standard error, p-value, confidence interval

[Andrea Baldin]

Dear all,
I am using the gsem command to estimate a latent class model. My sample size is quite small (around 250 observations) and the variables used are all binary.
My problem is that for some dummies I have the value of the coefficient but not the values of st.error, p-value and confidence interval. Is there a reason for this?
Thank you

[Weiwen Ng]

If multiple SEs are missing, then I suspect the model didn’t fully converge. You said that all indicators are binary. I would guess that several of them are at the boundary of the parameter space - that is, they are trending off to +/- infinity, which corresponds to a probability of 1 or 0.

In Stata, the solution is impose constraints on the parameters, as I’ve posted elsewhere. If you impose too many constraints, that can be a sign that you are trying to over-extract latent classes. There’s some discussion here. If it’s one of the intercepts in the multinomial model that’s missing an SE, then you are definitely trying to over-extract, and I would discard that solution. Remember, that situation is like saying that the probability of being in one of the latent classes is 0.

I didn’t discuss in that post, but I since remembered that some LCA softwares have the option to impose a non informative prior on the probabilities, even though they are not doing full Bayesian estimation. The Penn State University LCA package for Stata has this option. If you’re reading their manual, you are looking for the rho priors (their notation). I don't know how widespread this type of option is; I know that the R packages poLCA and flexmix don’t have this option, and base Stata doesn’t either.

[Andrea Baldin]

If multiple SEs are missing, then I suspect the model didn’t fully converge. You said that all indicators are binary. I would guess that several of them are at the boundary of the parameter space - that is, they are trending off to +/- infinity, which corresponds to a probability of 1 or 0.

In Stata, the solution is impose constraints on the parameters, as I’ve posted elsewhere. If you impose too many constraints, that can be a sign that you are trying to over-extract latent classes. There’s some discussion here. If it’s one of the intercepts in the multinomial model that’s missing an SE, then you are definitely trying to over-extract, and I would discard that solution. Remember, that situation is like saying that the probability of being in one of the latent classes is 0.

I didn’t discuss in that post, but I since remembered that some LCA softwares have the option to impose a non informative prior on the probabilities, even though they are not doing full Bayesian estimation. The Penn State University LCA package for Stata has this option. If you’re reading their manual, you are looking for the rho priors (their notation). I don't know how widespread this type of option is; I know that the R packages poLCA and flexmix don’t have this option, and base Stata doesn’t either.

Thank you Weiwen for your reply. In my context, there is just one variable than in one class has this problem. I thought Stata signals when there is not convergence, but in my case the model seems running. I suspect that, as there are few observations, there are no observations in that class for which the dummy variable (the one in which there is not confidence interval, p-value etc.) is equal to 1 (so probability 0). Does this speculation make sense?

[Weiwen Ng]

These days, I've switched to doing my initial class enumeration with 100 random starts and the secondary convergence criterion turned off (i.e. use the nonrtolerance option). With that option, Stata will declare convergence if any logit intercept hits +/-15 or exceeds that. When I'm done with all the random starts for that value of k (i.e. for that number of latent classes), I'll inspect the highest consistently converged LL value, then save its parameters as start values and have Stata maximize with its usual convergence criteria (i.e. no nonrtolerance option). If I need to constrain parameters, I do so at this step.

If you start your class enumeration with the nonrtolerance option off, then I think that Stata may sometimes or usually declare convergence if one logit intercept hits +/- 15 (but I am pretty sure the SE will return as missing). I am not sure about this, but with 2 or more logit intercepts at the boundary, Stata may refuse to converge.

Thank you Weiwen for your reply. In my context, there is just one variable than in one class has this problem... I suspect that, as there are few observations, there are no observations in that class for which the dummy variable (the one in which there is not confidence interval, p-value etc.) is equal to 1 (so probability 0). Does this speculation make sense?

Actually, that was not what I was talking about. Let's review. If you only have binary items, then the logit intercepts for each class are the class-specific probabilities of endorsing the indicator. It is possible for one of those probabilities to be 0 or 1. I am not sure if this is common in published literature, but it is certainly not unknown. Many people post to the forum with this issue. If you look at Masyn's chapter on LCA models that Stata's example cites, check out table 25.8 in her demonstration LCA model with 5 classes. The probability of endorsing the reverse coded math anxiety item (i.e. I do not experience anxiety about math problems) in latent class 1 is reported as 1.00 - that means that one intercept was either constrained at its boundary (+15) or the ML algorithm terminated normally with that intercept pretty close to 15. Or if you check my post 6 in the thread I referenced earlier, that paper may have constrained as many as 7 out of 10 indicators at their boundary values (I report 6 in that post, but one indicator actually appears to be at a probability of 1 as well). Anyway, one indicator constrained in one class is unlikely to be a problem. I would just examine the pattern of responses for internal consistency to be sure - I argued in post 6 that the latent class I was questioning seems to fail the internal consistency test.

Now, recall that you also have a multinomial model in LCA. In most LCA models, it's k-1 intercepts only, where k is the number of latent classes. This controls the probability of belonging to each latent class - remember that the LCA likelihood function for each latent class is the product of the probability of responses to each indicator, and the overall likelihood function is the product of the probability of belonging to each latent class. How do you determine that probability? With a multinomial logit model (to which you can add covariates, but you haven't got those). Anyway, the intercepts are at the very head of the standard output table. If one of those have a missing SE and a very low value, that's what would correspond to the posterior probability of belonging to that latent class being essentially 0. I have not yet had this happen to me yet. If it did, that would be a signal to discard that particular solution entirely. If the probability of being in one of the latent classes is 0, then I'm not sure I could trust the MLE of the parameters in that class ... but it would be moot, because the model would be telling me that nobody is even in that class.

[Andrea Baldin]

These days, I've switched to doing my initial class enumeration with 100 random starts and the secondary convergence criterion turned off (i.e. use the nonrtolerance option). With that option, Stata will declare convergence if any logit intercept hits +/-15 or exceeds that. When I'm done with all the random starts for that value of k (i.e. for that number of latent classes), I'll inspect the highest consistently converged LL value, then save its parameters as start values and have Stata maximize with its usual convergence criteria (i.e. no nonrtolerance option). If I need to constrain parameters, I do so at this step.

If you start your class enumeration with the nonrtolerance option off, then I think that Stata may sometimes or usually declare convergence if one logit intercept hits +/- 15 (but I am pretty sure the SE will return as missing). I am not sure about this, but with 2 or more logit intercepts at the boundary, Stata may refuse to converge.

Thank you Weiwen! Actually I realized I used the nonrtolerance option, that's way Stata didn't declare the non-convergence. Can I ask you how do you save the parameters as start values?

Anyway, coming to the second part of the discussion, the class specific intercept have their parameters estimated. The problem with missing SE concerns one specific dummy variable (in this case "going to the theatre in the last 12 months") in Class 2. What I meant is: if this variable has a value of -18 (obtained with the nonrtolerance option) and missing SE, can it be interpreted as the fact that there are NO people that have been to the theatre in the last 12 months in Class 2? (so the probability to have been to the theatre and belong to Class 2 is 0?

I am sorry I can't show the output but I'm working in a national statistics server and I can't export anything.

[Weiwen Ng]

Thank you Weiwen! Actually I realized I used the nonrtolerance option, that's way Stata didn't declare the non-convergence. Can I ask you how do you save the parameters as start values?

Pseudocode:

gsem (... <- , logit), lclass(C 5) startvalues(randomid(draws(100))) nonrtolerance
matrix b = e(b)
gsem (... <- , logit), lclass(C 5) from(b)

It's that simple. e(b) is the matrix of parameter estimates from the last estimation command; you have to save it as a permanent matrix before you can pass it to another round of the gsem command. Be aware also of the estimates store and estimates save commands. The former stores to memory, the latter to disk (in your current working directory). Often I have to come back to a previous model, so I prefer to save it permanently. After either command, you can type estimates restore or estimates use, then do the same routine to save the start values.

Anyway, coming to the second part of the discussion, the class specific intercept have their parameters estimated. The problem with missing SE concerns one specific dummy variable (in this case "going to the theatre in the last 12 months") in Class 2. What I meant is: if this variable has a value of -18 (obtained with the nonrtolerance option) and missing SE, can it be interpreted as the fact that there are NO people that have been to the theatre in the last 12 months in Class 2? (so the probability to have been to the theatre and belong to Class 2 is 0?

I am sorry I can't show the output but I'm working in a national statistics server and I can't export anything.

This is correct. To verify, type invlogit(-18) to get the probability that corresponds to a logit intercept of -18. For that matter, if you have an LCA model with only binary indicators and you only want point estimates of the class-specific probabilities, you can just inverse logit every logit intercept in your model. Some day, try running estat lcprob and comparing the point estimates from that to results from taking inverse logits.

As far as I know, MPlus will constrain parameters at 15 or -15. Type invlogit(15) or invlogit(-15) and you'll see that the result is still basically 0. So, I think it's sufficient to constrain at +/- 15. My instinctive preference would be to report how many papers you constrained if this were something peer-reviewed by an academic audience. However, I am not aware of hard and fast rules on this. This can be a bit of an obscure issue.