stata 14 meologit Refining starting values

[Gregory Saxton]

Hello,
I was trying to rerun a model using the meologit command in Stata 14, and I am getting completely different results than I was when I ran the model 3 months ago. Neither the syntax in my do-file, nor my data has changed since I last ran this model. When I compared the log file from three months ago to the output I'm currently getting, everything is exactly the same until I get to the "refining starting vales" line. Here, I notice that stata is giving me different numbers of iterations and different log likelihoods. I've pasted the 2 different model fitting outputs below. I'm concerned because an interaction I'm interested in was highly significant when I ran this model in January, and it's not longer significant at all. Does anyone have an idea what's going on? Any advice would be greatly appreciated.

Output from January 6, 2016
. meologit supDEM c.logGDPpc##i.unfair2 c.freedom##i.unfair2 swiid Education tr > usting rural persecon age govapprove quintall female SESsteps || country:,

Fitting fixed-effects model:
Iteration 0: log likelihood = -16371.452
Iteration 1: log likelihood = -15886.202
Iteration 2: log likelihood = -15882.538
Iteration 3: log likelihood = -15882.536
Iteration 4: log likelihood = -15882.536

Refining starting values:
Grid node 0: log likelihood = .
Grid node 1: log likelihood = -15990.742
Grid node 2: log likelihood = .
Grid node 3: log likelihood = .

Fitting full model:
Iteration 0: log likelihood = -15990.742
Iteration 1: log likelihood = -15987.619
Iteration 2: log likelihood = -15986.947
Iteration 3: log likelihood = -15986.945

Mixed-effects ologit regression

Number of obs = 14,790
Group variable: country
Number of groups = 18
Obs per group:
min = 556
avg = 821.7
max = 1,024

Integration method: mvaghermite
Integration pts. = 7
Wald chi2(15) = 752.93 Log likelihood = -15986.945 Prob > chi2 = 0.0000


Output from March 24, 2016

. meologit supDEM c.logGDPpc##i.unfair2 c.freedom##i.unfair2 swiid Education trusting rural persecon age govapprove quintall female SESsteps || country:,

Fitting fixed-effects model:

Iteration 0: log likelihood = -16371.452
Iteration 1: log likelihood = -15886.202
Iteration 2: log likelihood = -15882.538
Iteration 3: log likelihood = -15882.536
Iteration 4: log likelihood = -15882.536

Refining starting values:

Grid node 0: log likelihood = -15549.873

Fitting full model:

Iteration 0: log likelihood = -15549.873 (not concave)
Iteration 1: log likelihood = -15547.201 (not concave)
Iteration 2: log likelihood = -15544.544 (not concave)
Iteration 3: log likelihood = -15542.294 (not concave)
Iteration 4: log likelihood = -15541.411
Iteration 5: log likelihood = -15521.021
Iteration 6: log likelihood = -15513.002
Iteration 7: log likelihood = -15512.794
Iteration 8: log likelihood = -15512.793

Mixed-effects ologit regression
Number of obs = 14,790
Group variable: country
Number of groups = 18

Obs per group:
min = 556
avg = 821.7
max = 1,024

Integration method: mvaghermite
Integration pts. = 7

Wald chi2(15) = 578.65
Log likelihood = -15512.793
Prob > chi2 = 0.0000

[Steve Samuels]

Welcome to Statalist, Gregory!

You don't say whether you updated Stata, nor what version you are using. However your most recent model has a higher log-likelihood; thus the earlier run did not produce the maximum likelihood estimates. The algorithm, if you are using Stata MP, has indeed changed. This is output from help whatsnew:

-- update 03mar2016 ---------------------------------------------------------------------------

2. Stata/MP performance has improved, often dramatically, for the following commands: gsem,
irt, meglm, melogit, meprobit, mecloglog, meologit, meoprobit, mepoisson, menbreg,
mestreg, xtologit, xtoprobit, and xtstreg.

In future posts, please put commands and results between CODE delimiters, described in FAQ 12.

[Gregory Saxton]

Hello,
Thanks for the response. I am now using Stata 14.1 IC. Is there any way to run the model in a way that will produce the same output as I was getting with Stata 14 (i.e. prior to the update).
Best,
Greg Saxton

[Steve Samuels]

There's no way that I know of to get the previous output. These are tough models to fit and that output, judging by the log-likelihoods, was seriously incorrect.

Interactions for these models appear very difficult to interpret; I suggest that you use margins to look at results on the probability scale.

[Steve Samuels]

I notice the following about meologit in the January 28 update:

15. The multilevel mixed-effects estimators gsem, meglm, melogit, meprobit, meologit,
meoprobit, mepoisson, menbreg, and mestreg have the following improvement and fix:

a. gsem, meglm, melogit, meprobit, meologit, meoprobit, mepoisson, menbreg, and mestreg
are now more likely to converge for models fit to data with large group sizes.

b. gsem, meglm, melogit, meprobit, meologit, meoprobit, mepoisson, menbreg, and mestreg,
when used with datasets that had many observations per group, could indicate that the
model converged and report results that did not include the large groups in the
computations. To determine whether your model was affected, you can use predict to
obtain empirical Bayes mean estimates for the random effects and check for zero values
in large groups. This has been fixed.

Your group sizes range from 556 to 1,024. If this interval contains sizes that triggered the fixed error, that would explain the difference that you are seeing. For more light on the question, I suggest that you contact Technical support [email protected].

[Evan Fuller]

I'm sure this has been resolved or no longer is an issue for the people involved in this thread, but for the sake of those who may encounter a similar problem in the future, I would like to mention that the "version" command

(https://www.stata.com/manuals13/pversion.pdf)

will allow code to be run under previous versions in order to produce estimates which are consistent with prior runs. However, as the estimates changed in this case due to an improvement in the algorithm used in maximum likelihood estimation, it is not desireable to use the prior version result just to p-hack an estimated coefficient to achieve some desired level of significance.