How do xtlogit and melogit differ?

[William Lisowski]

I am fitting a logistic model with random effects to panel data. For my data, xtlogit ..., re and melogit ... || panel_id: yield essentially identical results. The only noticeable differences is that xtlogit takes painfully longer to run.

Can someone point me to a discussion of the differences between the approaches embodied by the two procedures? I expect it is at a "philosophical" level, not just different pragmatic choices in the coding of the two procedures.

Since I'm analyzing panel data, xtlogit seems the natural choice for simple random effects - I'm not looking at a more complicated mixed model that only melogit can handle.. I'm reassured that for this simple case, xtlogit and melogit yield essentially identical results.

My interest is to learn a little more about these two approaches to the same problem, and to understand any differences in the underlying assumptions. I'm hoping there's something already written that I didn't find in searching Statalist and using Google to search more widely.

Thanks!

[Rich Goldstein]

I think of it like this: xtlogit allows a random effect for the constant and this random effect does not depend (is not associated with) any other variables; melogit relaxes both of those constraints: that is, you can have random slopes as well as a random intercept and you can have the random effect "depend on" on other variables (the RE equation); not sure if this is the kind of answer you are looking for

[Enrique Pinzon (StataCorp)]

Hello William,

The models been fit in your case are the same. Also, the method used, Gauss–Hermite quadrature, is the same. The code, however, is a bit different, mainly, in terms of the starting values and the number of quadrature points being used. This is partly because melogit allows for more complex models, as the ones mentioned by Richard. It is unclear to me exactly what you mean by painfully slow. My intuition is that given the same starting values and number of quadrature points xtlogit should be a bit quicker.

[William Lisowski]

Enrique's response mirrors my expectations. However, my experience differs. Below I run the first example from [XT] xtlogit using both xtlogit and melogit with timers, using Stata/SE 14.1 for Mac (64-bit Intel) Revision 30 Mar 2016 on a 2015 MacBook (no Pro, no Air).

Let me first ask, please review my melogit command: did I miss a necessary option on melogit necessary to make it comparable to xtlogit?

Comments on the output of this example:

1. xtlogit consistently runs about 16 seconds; melogit about 4. (For my actual dataset, I have not set timers, but the times for a group of three runs are minutes, not seconds, and the differences are easily noticeable.) I've run the two commands in both orders to make sure caching and the like wasn't affecting the comparison; the speeds are similar in either order.
2. There are noticeable differences in the results of this test. In my actual dataset, the differences were negligible.
3. xtlogit includes a sequence of values of tau at the start of fitting the full model; not so melogit; this suggests there are some differences under the hood.
4. xtlogit used 12 integration points fitting the full model; melogit 7; this is consistent with what Enrique wrote.
5. ereturn list results show e(ml_method) of "d2" for xtlogit and "gf2" for melogit; there are several other differences of this type; again suggesting that something may be different under the hood, or I've not set up the melogit test correctly.

Below is the code I ran from the do-file editor for my test.

Added in edit: adding intpoints(7) to xtlogit did not materially affect the speed but did make some of the results more comparable. Specifically, the final log likelihood on the full model increased (became a smaller negative number) and was closer to that for melogit.

cls
set more off
timer clear
use http://www.stata-press.com/data/r14/union, clear
describe, short
xtset
timer on 1
xtlogit union age grade not_smsa south##c.year, re
timer off 1
ereturn list
timer on 2
melogit union age grade not_smsa south##c.year || idcode:
timer off 2
ereturn list
timer list
set more on

[Rich Goldstein]

your xtset command is incomplete - it appears that it should be "xtset idcode"

[William Lisowski]

Sorry for the confusion: my argumentless xtset was to report on the file's existing setting, to confirm what I needed as the random effect in melogit.

[Enrique Pinzon (StataCorp)]

Hi William,

I have looked into what you reported and in my Unix and Windows machines xtlogit runs faster. I am using Stata MP in both of them. However, once I set processors 1 , which is equivalent to having Stata SE, I can replicate the behavior you are mentioning. I will look into it and let you know more details about this behavior.

[Enrique Pinzon (StataCorp)]

Hello William,

I have looked into what is happening and the executive summary is that the evaluators used during the optimization process are using different computations. This is what your intuition suggested. The results do not seem to be affected by starting values (they use the same starting values except for the variance component) nor by the optimization method (gf2 vs d2). The differences arise in the intra-panel calculations. As your case indicates, this is true for Stata SE but the differences disappear when you use multiprocessor Stata flavors. We are looking into ways to optimize the computations of xtlogit for Stata SE.

[William Lisowski]

Hello Enrique -

Many thanks for your efforts researching this issue. I have one question to confirm my understanding.

At the level of the statistical theory are the simple commands I tested fitting the same model with the same assumptions and thus estimating the same quantities?

It was the substantial difference in runtime, along with small differences in reported results, that led me to think that the code was different, and I the guessed that the difference was due to different underlying assumptions in xtlogit, re and melogit. That seems to have a bad guess, and it led to the question I posed in my first post.

I think what I am hearing is that differences in code reflect only differences in capabilities, not differences in the underlying statistical approach; and that where the same model can be fit with either command any differences in the reported results only reflects the different numerical approaches. That would explain why I haven't found a discussion of the differences between the two commands: there is no essential difference.

Beyond that, while increased efficiency in xtlogit is certainly to be desired, knowing that both are aiming for the same target makes me more confident in substituting melogit for xtlogit, re if the xtlogit run times become onerous.

[Enrique Pinzon (StataCorp)]

Hello William,

This is true for the -re- commands that have an -xt- and -me- counterpart. The differences are numerical as you point out.

[George Hoffman]

Hello William,

I have looked into what is happening and the executive summary is that the evaluators used during the optimization process are using different computations. This is what your intuition suggested. The results do not seem to be affected by starting values (they use the same starting values except for the variance component) nor by the optimization method (gf2 vs d2). The differences arise in the intra-panel calculations. As your case indicates, this is true for Stata SE but the differences disappear when you use multiprocessor Stata flavors. We are looking into ways to optimize the computations of xtlogit for Stata SE.

i have StataNow on win64 and would report that xtlogit runs many times more slowly than melogit. is there an update on the status of optimizatoin of xt, re approaches for StataSE? or should i migrate to the me- command set more generally?
thx!