Help with recursive estimation FMM Stata 15

[Alejandro Gonzalez]

Hi everybody.

I'm currently estimating a Finite Mixture Model (FMM) with Stata 15. My model is of the form:

fmm 2, lcprob(z1 z2): regress y1 x1 x2

My model has way more covariates, but the syntax is to set ideas.

What I want to do is the following:

1. Estimate the model

fmm 2: regress y1 x1 x2

2. Store the values of the parameters, both of x1 and x2 for each regime, and for the latent class probabilities.

3. Estimate the model

fmm 2, lcprob(z1 z2): regress y1 x1 x2

But using as initial values for the class probabilities and for the parameters the output of the previous regressions (which I stored in step 2).

Could anyone help me on this? FMM has an option of the form startv(g1 g2), where g1 and g2 are variables that contain the class probabilities for each observation. I tried running this option, but the log-likelihood reports a discontinuity in the second iteration. Besides, this only allows me to set as initial values the class probabilities, and not the values of the parameters x1 and x2, which I also want.

Thanks!
Alejandro.

[Weiwen Ng]

...

1. Estimate the model

fmm 2: regress y1 x1 x2

2. Store the values of the parameters, both of x1 and x2 for each regime, and for the latent class probabilities.

3. Estimate the model

fmm 2, lcprob(z1 z2): regress y1 x1 x2

But using as initial values for the class probabilities and for the parameters the output of the previous regressions (which I stored in step 2).

....

A couple issues. The syntax you typed on the forum doesn't appear complete if your goal was to obtain start values using each observation's class ID set probabilistically. I'm not sure what exactly the command as typed does. However, if you want to do this, I believe the correct syntax is:

fmm 2: regress y1 x1 x2
predict pclass*, classposteriorpr
fmm 2, lcprob(z1 z2) startvalues(classpr pclass1 pclass2): regress y1 x1 x2

The manual says that starting values specified using the -from- option will override the computed start values. I'm not 100% sure how the options interact. However, I can show you with a stock Stata dataset that when you specify the -classpr- option in your context, the parameter start values should be identical to the ones you just fit in the prior model. In the syntax below, the -emopts(iterate(0))- option tells Stata not to do any EM iterations, and the -noestimate- option tells Stata not to do any iterations with its usual maximizer. Hence, when I store the estimates from the second block of code below, that is going to reflect the parameter values that Stata would have started estimating from.

webuse mus03sub
fmm 3: regress lmedexp income c.age##c.age totchr i.sex
estimates store fmm3_v1
predict pclass*, classposteriorpr

fmm 3, startvalues(classpr pclass*) lcprob(totchr) noestimate emopts(iterate(0)): regress lmedexp income c.age##c.age totchr i.sex
estimates store fmm3_v2
estimates table fmm3*

The multinomial models (which estimate class probability) obviously differ between the two saved estimates, but we were expecting that. All the class-specific parameter values are nearly identical - they correspond to at least 4 decimal places, as far as I can see. Hence, specifying the class probabilities as start values should be sufficient to replicate the starting values for your parameters without any further input.

Try that and see if you still get the likelihood discontinuous error message. Hopefully this fixes it.

[Weiwen Ng]

...

webuse mus03sub
fmm 3: regress lmedexp income c.age##c.age totchr i.sex
estimates store fmm3_v1
predict pclass*, classposteriorpr

fmm 3, startvalues(classpr pclass*) lcprob(totchr) noestimate emopts(iterate(0)): regress lmedexp income c.age##c.age totchr i.sex
estimates store fmm3_v2
estimates table fmm3*

...

For everyone's reference, here are the results of the -estimates table- command.

. estimates table fmm3*

----------------------------------------
    Variable |  fmm3_v1      fmm3_v2    
-------------+--------------------------
1b.Class     |
      totchr |               (omitted)  
       _cons |  (omitted)    (omitted)  
-------------+--------------------------
2.Class      |
      totchr |               .06159938  
       _cons |  1.1622956    1.0528741  
-------------+--------------------------
3.Class      |
      totchr |                .0933005  
       _cons |  -1.153202   -1.3214146  
-------------+--------------------------
lmedexp      |
       Class#|
    c.income |
          1  |  .00598043    .00598044  
          2  |  .00237254    .00237254  
          3  |  .00093152     .0009315  
             |
 Class#c.age |
          1  |   .1201823    .12018218  
          2  |  .21366582    .21366545  
          3  |  -.2645947   -.26459449  
             |
 Class#c.age#|
       c.age |
          1  | -.00075724   -.00075724  
          2  | -.00131948   -.00131948  
          3  |  .00157613    .00157612  
             |
       Class#|
    c.totchr |
          1  |   .9223744    .92237415  
          2  |  .31065858    .31065853  
          3  |  .18647496    .18647472  
             |
   sex#Class |
   female#1  |  .05765083    .05765074  
   female#2  | -.09189238    -.0918925  
   female#3  | -.17614841   -.17614811  
             |
       Class |
          1  |  .63009655    .63010196  
          2  | -.95467213   -.95465742  
          3  |  20.795239     20.79523  
-------------+--------------------------
var(e.lmed~p)|
       Class |
          1  |  1.4318301    1.4318297  
          2  |   .7966127    .79661269  
          3  |  .38468907    .38468914  
----------------------------------------

Remember: fmm_v1 was a 3-class FMM model successfully fit with no covariates. fmm_v2 is a 3-class FMM model fit with one covariate (totchr), from starting values defined by the class probabilities found in the prior model, and I forced Stata to not run both the EM algorithm and the usual likelihood maximizer at all. Thus, fmm_v2 reflects the pure start values. They are all but equivalent to the parameter estimates in fmm_v1 except for the multinomial part of the FMM model.

Therefore, the -classpr- option alone should accomplish what Alejandro wants to do (i.e. fit an FMM model from starting parameter values and class probabilities found in an earlier one).

[Alejandro Gonzalez]

Hi,

Thanks! This works wonder. As a side note, I reccommend everybody to estimate FMM models sequentially, as done with this example. Fitting the whole model in one go using the EMM algorithm took me 36 hours, while with this method, it took roughly 25 minutes (as a side note, my compute is pretty up to date, i7 sixth generation, 8g RAM).

A quick note: The way I implemented my recursive estimation differed from yours in the following way:

fmm 2: regress y1 x1 x2
estat lcprob
ret list matrix
A = e(b)
gen g1 = A[1,1]
gen g2 = A[1,2]
fmm 2, lcprob(z1 z2): regress y1 x1 x2.

I noticed that the way you proposed it, the predicted class probabilities are NOT constant across observations. This is odd, given that in the baseline model (without modelling class probabilities), the class probabilities should be a fixed probability for each observation, given that we only insert the intercept. Thus, I'm not really sure why the option

predict pclass*, classposteriorpr

Shows different class probabilities for each observation in a model where the class probability is a constant. Cheers! Alejandro.

[Weiwen Ng]

Hi,

Thanks! This works wonder. As a side note, I reccommend everybody to estimate FMM models sequentially, as done with this example. Fitting the whole model in one go using the EMM algorithm took me 36 hours, while with this method, it took roughly 25 minutes (as a side note, my compute is pretty up to date, i7 sixth generation, 8g RAM).

A quick note: The way I implemented my recursive estimation differed from yours in the following way:

fmm 2: regress y1 x1 x2
estat lcprob
ret list matrix
A = e(b)
gen g1 = A[1,1]
gen g2 = A[1,2]
fmm 2, lcprob(z1 z2): regress y1 x1 x2.

I noticed that the way you proposed it, the predicted class probabilities are NOT constant across observations. This is odd, given that in the baseline model (without modelling class probabilities), the class probabilities should be a fixed probability for each observation, given that we only insert the intercept. Thus, I'm not really sure why the option

predict pclass*, classposteriorpr

Shows different class probabilities for each observation in a model where the class probability is a constant. Cheers! Alejandro.

Alejandro,

I am glad this worked out for you!

On your question about prior probabilities, I believe that the posterior probabilities of class membership in an FMM model without covariates do vary across observations. If you read through the FMM intro manual, on page 7, it says

We can use the predictions of the posterior probability of class membership to evaluate the probability of being in each class for each stamp.
... Because there are no covariates in the model, the posterior probabilities are the same for any stamp with a given thickness and are as follows.
... (followed by a table)

Consider this: even though you specified no covariates which influence the probability of class membership on the multinomial side of the equation, if I know your value of y, I have some guess as to your class membership. Say we somehow could not observe gender, but we have an FMM of human heights with no predictors on the multinomial side. If I know that someone is 6' 8" tall, then chances are good that they are male, right?

So, which set of start values should you use? You took the approach of using something like the prior probabilities of class membership (I am not formally trained in Bayesian methods and I am pretty sure I have mangled that term, but I am going to just use it in this example). I took the posterior probabilities. As long as the prior probabilities are a reasonable estimate, then they should improve the maximization process nonetheless. Chances are excellent that using either the prior or posterior probabilities as start values should wind up at an equivalent maximum likelihood estimate. I would assume that using the posterior probabilities could get you there faster.