Error with lrtest after melogit

Michael Evangelist

Join Date: Mar 2017
Posts: 10

Error with lrtest after melogit

11 Jun 2018, 13:41

I am trying to figure out why I receive an error after using lrtest to compare two models following melogit. In the first model, I include age as a categorical variable. In the second model, I include age and age squared as continuous variables. The second model should be nested in the first, but I receive error r(498) "Mixed models are not nested." Interestingly, I do not receive an error message when I use mixed or xtmelogit or when I exclude age-squared from the second model or when I use dummy variables instead of factor variables in the first model.

Here is a reproducible example of the error using a demo of the high school and beyond data. I do not have the ability to update Stata.

Stata/SE 15.0 for Windows (64-bit x86-64)
Revision 06 Jun 2017

Code:

use "https://stats.idre.ucla.edu/stat/data/hsbdemo.dta", clear

keep id cid honors

* generate age (0 to 5)
gen age = floor(6*runiform() + 0)
tab age, gen(age)

* (1) xtmelogit
xtmelogit honors i.age || cid:, nolog
estimates store m1

xtmelogit honors c.age##c.age || cid:, nolog
lrtest m1

* (2) mixed
mixed honors i.age || cid:, nolog
estimates store m1

mixed honors c.age##c.age || cid:, nolog
lrtest m1

* (3) melogit (no factor variables)

melogit honors age2-age6 || cid:, nolog
estimates store m1

melogit honors c.age##c.age || cid:, nolog
lrtest m1

* (4) melogit
melogit honors i.age || cid:, nolog
estimates store m1

melogit honors c.age##c.age || cid:, nolog
lrtest m1

Code:

. * (1) xtmelogit
. xtmelogit honors i.age || cid:, nolog

Mixed-effects logistic regression               Number of obs     =        200
Group variable: cid                             Number of groups  =         20

                                                Obs per group:
                                                              min =          7
                                                              avg =       10.0
                                                              max =         12

Integration points =   7                        Wald chi2(5)      =      12.44
Log likelihood = -74.701593                     Prob > chi2       =     0.0293

------------------------------------------------------------------------------
      honors |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         age |
          1  |  -.1733401   .8012485    -0.22   0.829    -1.743758    1.397078
          2  |  -.3941871   .8879459    -0.44   0.657    -2.134529    1.346155
          3  |   .2470597   .9129151     0.27   0.787    -1.542221     2.03634
          4  |   2.084026   .9343249     2.23   0.026     .2527832    3.915269
          5  |  -1.498702   .9218457    -1.63   0.104    -3.305487    .3080821
             |
       _cons |  -2.475971   1.018342    -2.43   0.015    -4.471884   -.4800577
------------------------------------------------------------------------------

------------------------------------------------------------------------------
  Random-effects Parameters  |   Estimate   Std. Err.     [95% Conf. Interval]
-----------------------------+------------------------------------------------
cid: Identity                |
                   sd(_cons) |   2.913941   .7780829      1.726602    4.917783
------------------------------------------------------------------------------
LR test vs. logistic model: chibar2(01) = 68.71       Prob >= chibar2 = 0.0000

. estimates store m1

. 
. xtmelogit honors c.age##c.age || cid:, nolog

Mixed-effects logistic regression               Number of obs     =        200
Group variable: cid                             Number of groups  =         20

                                                Obs per group:
                                                              min =          7
                                                              avg =       10.0
                                                              max =         12

Integration points =   7                        Wald chi2(2)      =       1.31
Log likelihood = -82.427681                     Prob > chi2       =     0.5205

------------------------------------------------------------------------------
      honors |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         age |   .5909583   .5243571     1.13   0.260    -.4367627    1.618679
             |
 c.age#c.age |  -.1140369   .1002671    -1.14   0.255    -.3105567     .082483
             |
       _cons |  -2.449259   .8802882    -2.78   0.005    -4.174593   -.7239263
------------------------------------------------------------------------------

------------------------------------------------------------------------------
  Random-effects Parameters  |   Estimate   Std. Err.     [95% Conf. Interval]
-----------------------------+------------------------------------------------
cid: Identity                |
                   sd(_cons) |    2.56678   .6665902      1.542881    4.270167
------------------------------------------------------------------------------
LR test vs. logistic model: chibar2(01) = 66.12       Prob >= chibar2 = 0.0000

. lrtest m1

Likelihood-ratio test                                 LR chi2(3)  =     15.45
(Assumption: . nested in m1)                          Prob > chi2 =    0.0015

. 
. * (2) mixed
. mixed honors i.age || cid:, nolog

Mixed-effects ML regression                     Number of obs     =        200
Group variable: cid                             Number of groups  =         20

                                                Obs per group:
                                                              min =          7
                                                              avg =       10.0
                                                              max =         12

                                                Wald chi2(5)      =      17.22
Log likelihood = -70.654349                     Prob > chi2       =     0.0041

------------------------------------------------------------------------------
      honors |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         age |
          1  |  -.0189632   .0776508    -0.24   0.807    -.1711559    .1332296
          2  |  -.0497185   .0807616    -0.62   0.538    -.2080084    .1085713
          3  |   .0015014   .0853372     0.02   0.986    -.1657563    .1687592
          4  |   .1738509   .0778137     2.23   0.025     .0213389    .3263628
          5  |  -.1424887   .0811943    -1.75   0.079    -.3016266    .0166493
             |
       _cons |   .2659201   .0873002     3.05   0.002     .0948147    .4370254
------------------------------------------------------------------------------

------------------------------------------------------------------------------
  Random-effects Parameters  |   Estimate   Std. Err.     [95% Conf. Interval]
-----------------------------+------------------------------------------------
cid: Identity                |
                  var(_cons) |   .0878426   .0307969      .0441856    .1746342
-----------------------------+------------------------------------------------
               var(Residual) |   .0939999   .0099082      .0764549    .1155713
------------------------------------------------------------------------------
LR test vs. linear model: chibar2(01) = 85.92         Prob >= chibar2 = 0.0000

. estimates store m1

. 
. mixed honors c.age##c.age || cid:, nolog

Mixed-effects ML regression                     Number of obs     =        200
Group variable: cid                             Number of groups  =         20

                                                Obs per group:
                                                              min =          7
                                                              avg =       10.0
                                                              max =         12

                                                Wald chi2(2)      =       1.39
Log likelihood = -78.212003                     Prob > chi2       =     0.4986

------------------------------------------------------------------------------
      honors |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         age |   .0604468   .0523612     1.15   0.248    -.0421793    .1630728
             |
 c.age#c.age |  -.0117794   .0100052    -1.18   0.239    -.0313891    .0078304
             |
       _cons |   .2235274     .08666     2.58   0.010     .0536768    .3933779
------------------------------------------------------------------------------

------------------------------------------------------------------------------
  Random-effects Parameters  |   Estimate   Std. Err.     [95% Conf. Interval]
-----------------------------+------------------------------------------------
cid: Identity                |
                  var(_cons) |   .0917306   .0322449      .0460577    .1826948
-----------------------------+------------------------------------------------
               var(Residual) |   .1017038   .0107192      .0827225    .1250405
------------------------------------------------------------------------------
LR test vs. linear model: chibar2(01) = 83.66         Prob >= chibar2 = 0.0000

. lrtest m1

Likelihood-ratio test                                 LR chi2(3)  =     15.12
(Assumption: . nested in m1)                          Prob > chi2 =    0.0017

. 
. * (3) melogit (no factor variables)
. 
. melogit honors age2-age6 || cid:, nolog

Mixed-effects logistic regression               Number of obs     =        200
Group variable:             cid                 Number of groups  =         20

                                                Obs per group:
                                                              min =          7
                                                              avg =       10.0
                                                              max =         12

Integration method: mvaghermite                 Integration pts.  =          7

                                                Wald chi2(5)      =      12.56
Log likelihood = -74.635274                     Prob > chi2       =     0.0278
------------------------------------------------------------------------------
      honors |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
        age2 |  -.1720487   .8026755    -0.21   0.830    -1.745264    1.401166
        age3 |  -.3887137   .8896556    -0.44   0.662    -2.132407    1.354979
        age4 |   .2527461     .91495     0.28   0.782    -1.540523    2.046015
        age5 |   2.089703   .9305742     2.25   0.025     .2658107    3.913594
        age6 |  -1.499533   .9232011    -1.62   0.104    -3.308974    .3099077
       _cons |  -2.535821   1.009193    -2.51   0.012    -4.513803   -.5578385
-------------+----------------------------------------------------------------
cid          |
   var(_cons)|   8.954448   5.084708                      2.942328    27.25125
------------------------------------------------------------------------------
LR test vs. logistic model: chibar2(01) = 68.84       Prob >= chibar2 = 0.0000

. estimates store m1

. 
. melogit honors c.age##c.age || cid:, nolog

Mixed-effects logistic regression               Number of obs     =        200
Group variable:             cid                 Number of groups  =         20

                                                Obs per group:
                                                              min =          7
                                                              avg =       10.0
                                                              max =         12

Integration method: mvaghermite                 Integration pts.  =          7

                                                Wald chi2(2)      =       1.32
Log likelihood = -82.366054                     Prob > chi2       =     0.5169
------------------------------------------------------------------------------
      honors |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         age |   .5949913   .5249916     1.13   0.257    -.4339734    1.623956
             |
 c.age#c.age |   -.114767   .1003773    -1.14   0.253    -.3115028    .0819689
             |
       _cons |  -2.490561   .8634145    -2.88   0.004    -4.182823   -.7983003
-------------+----------------------------------------------------------------
cid          |
   var(_cons)|   6.930176   3.823084                      2.350566    20.43224
------------------------------------------------------------------------------
LR test vs. logistic model: chibar2(01) = 66.25       Prob >= chibar2 = 0.0000

. lrtest m1

Likelihood-ratio test                                 LR chi2(3)  =     15.46
(Assumption: . nested in m1)                          Prob > chi2 =    0.0015

. 
. * (4) melogit
. melogit honors i.age || cid:, nolog

Mixed-effects logistic regression               Number of obs     =        200
Group variable:             cid                 Number of groups  =         20

                                                Obs per group:
                                                              min =          7
                                                              avg =       10.0
                                                              max =         12

Integration method: mvaghermite                 Integration pts.  =          7

                                                Wald chi2(5)      =      12.56
Log likelihood = -74.635274                     Prob > chi2       =     0.0278
------------------------------------------------------------------------------
      honors |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         age |
          1  |  -.1720487   .8026755    -0.21   0.830    -1.745264    1.401166
          2  |  -.3887137   .8896556    -0.44   0.662    -2.132407    1.354979
          3  |   .2527461     .91495     0.28   0.782    -1.540523    2.046015
          4  |   2.089703   .9305742     2.25   0.025     .2658107    3.913594
          5  |  -1.499533   .9232011    -1.62   0.104    -3.308974    .3099077
             |
       _cons |  -2.535821   1.009193    -2.51   0.012    -4.513803   -.5578385
-------------+----------------------------------------------------------------
cid          |
   var(_cons)|   8.954448   5.084708                      2.942328    27.25125
------------------------------------------------------------------------------
LR test vs. logistic model: chibar2(01) = 68.84       Prob >= chibar2 = 0.0000

. estimates store m1

. 
. melogit honors c.age##c.age || cid:, nolog

Mixed-effects logistic regression               Number of obs     =        200
Group variable:             cid                 Number of groups  =         20

                                                Obs per group:
                                                              min =          7
                                                              avg =       10.0
                                                              max =         12

Integration method: mvaghermite                 Integration pts.  =          7

                                                Wald chi2(2)      =       1.32
Log likelihood = -82.366054                     Prob > chi2       =     0.5169
------------------------------------------------------------------------------
      honors |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         age |   .5949913   .5249916     1.13   0.257    -.4339734    1.623956
             |
 c.age#c.age |   -.114767   .1003773    -1.14   0.253    -.3115028    .0819689
             |
       _cons |  -2.490561   .8634145    -2.88   0.004    -4.182823   -.7983003
-------------+----------------------------------------------------------------
cid          |
   var(_cons)|   6.930176   3.823084                      2.350566    20.43224
------------------------------------------------------------------------------
LR test vs. logistic model: chibar2(01) = 66.25       Prob >= chibar2 = 0.0000

. lrtest m1
Mixed models are not nested
r(498);

Tags: None

Richard Williams

Join Date: Apr 2014

Posts: 4941
#2

11 Jun 2018, 20:36

I don't know why it is being fussy. But I changed the last command to

Code:

lrtest m1, force

and it worked. You should check that it worked correctly. You didn't set the seed so I couldn't exactly replicate your results.

-------------------------------------------
Richard Williams, Notre Dame Dept of Sociology
StataNow Version: 19.5 MP (2 processor)
EMAIL: [email protected]
WWW: https://www3.nd.edu/~rwilliam
Comment
Michael Evangelist

Join Date: Mar 2017

Posts: 10
#3

12 Jun 2018, 06:31

Richard, Thank you for this solution. It will save me from coding the test manually. I still wonder about the reason for the error message. From my experience, there's usually a good reason why Stata refuses to perform a given task, but in this case, I thought it might be a bug given that lrtest works fine with the equivalent dummy variable approach. Since you responded, I also wanted to say thanks for making your PowerPoint on margins available. I refer to the slides often and have recommended the presentation many times to colleagues.
1 like
Comment
Richard Williams

Join Date: Apr 2014

Posts: 4941
#4

12 Jun 2018, 10:11

I suppose it is possible the old behavior was considered buggy because it might let you contrast non-nested models. In any event, there are times when we know two models are nested and Stata doesn't, and that is why the force option exists.

Glad you like the margins handouts. They have proven more popular than I expected.

-------------------------------------------
Richard Williams, Notre Dame Dept of Sociology
StataNow Version: 19.5 MP (2 processor)
EMAIL: [email protected]
WWW: https://www3.nd.edu/~rwilliam
Comment
Rafal Raciborski (StataCorp)

StataCorp Employee

Join Date: Mar 2014

Posts: 83
#5

12 Jun 2018, 10:14

Michael found a bug in melogit. When covariates in the fixed-effects equation contain factor variables, melogit and related me commands miscount the number of parameters that is subsequently stored in e(k_f) and lrtest complains. We will have this fixed in a future update to Stata, in the meantime Michael can use lrtest, force as a workaround.
Comment
Michael Evangelist

Join Date: Mar 2017

Posts: 10
#6

12 Jun 2018, 10:26

Rafel, Thank you for looking into the issue. That's good to know.
Comment

Announcement

Error with lrtest after melogit

Comment

Comment

Comment

Comment

Comment