Why the result of different way to test the proportinal odds/parallel lines assumption of the ordered logit model is different?

[Xue Gao]

Hello!
I run the ordered logit model which have the parallel lines assumption.
I use brant command to test the parallel lines and it shows that overall chi-square value is insignificant which suggests that ologit’s assumptions are met (p>chi2=0.177).
I use omodel command and its p>chi2=0.0869.
I run the gologit2 command with autofit (using the .05 level of significance), it shows that constraints for parallel lines are not imposed for education variable and income variable.
What is the difference between these test?
What should i do? I say that brant test suggest the parallel lines are met and i run the ordered logit model? Or I say that the parallel lines assumption is always violated and so I use the gologit2 with autofit directly?
Can you help me if you know it? Thank you very much.

[Maarten Buis]

Most tests are approximations, so different approximations can give you different results. The p-values you gave are not that different. So I suspect that is what is going on. Rich Williams an I did some simulations to see how different tests of the proportional odds assumption behaves in different scenarios and presented the results here:

http://maartenbuis.nl/presentations/gsug13.pdf

I would say the results are just inconclusive. I suggest you just look at the coefficients from the constrained and unconstrained model and see if the differences are by your subjective judgment "big". If that is the case you can consider the unconstrained model, assuming you have sufficient observations, no sparse categories, etc.etc.

[Xue Gao]

Dear Maarten Buis,
My dependent variable have three degrees.In the data set above, I look at the coefficients from the constrained and unconstrained model. For example, the coefficient of education variable is 0.268 in ologit model, 0.185 and 0.447 in gologit model, 0.167 and 0.451 in gologit2 model with autofit.
In the meantime, I have other data sets and I test the parallel lines assumption. Sample size is between 650 and 3000 in each data set.
In one data set, the brant test shows that p>chi2=0.037; The omodel command shows that p>chi2=0.5746; The gologot2 with auotifit shows that all explanatory variables meet the parallel lines assumption.
In one data set, the brant test shows that p>chi2=0.005; The omodel command shows that p>chi2=0.6684; The gologot2 with auotifit shows that sex variables violated the parallel lines assumption.
In some data sets, all the tests show that the parallel lines assumption are met.
If I use the gologit2 with autofit directly in every data set, is it ok？

[Richard Williams]

It might help if we could see the actual commands and output.

If I could live my life over, I would use .01 as the default for gologit2's autofit or maybe something even stricter. When doing multiple tests, some are going to show up as significant just by chance.

As much as I like gologit2, if a reasonable case can be made for sticking with ologit I often think that is the better way to go. Conflicting test result or an examination of BIC and AIC tests might possibly be used to make the case for ologit.

[Xue Gao]

Hello!
I run the ordered logit model which have the parallel lines assumption.
I use brant command to test the parallel lines and it shows that overall chi-square value is insignificant which suggests that ologit’s assumptions are met (p>chi2=0.177).
I use omodel command and its p>chi2=0.0869.
I run the gologit2 command with autofit (using the .05 level of significance), it shows that constraints for parallel lines are not imposed for education variable and income variable.

The result is as follow. When i use 0.01 as the default of autofit, all the explanatory variables meet the parallel line assumption.
In this situation, what is the best solution?
（1）brant command.

(2) omodel command

(3) autofit command

(4) autofit(0.01) command

[Joseph Coveney]

In this situation, what is the best solution?

Probably only you can make that judgment call. Others can only offer advice. Along those lines, how about: "look at the coefficients from the constrained and unconstrained model and see if the differences are by your subjective judgment 'big'" and "if a reasonable case can be made for sticking with ologit I often think that is the better way to go"?

See also advice given in the latter half of this thread, including the assessment method that is referenced in one of the posts there.

With 650 to 3000 observations, eight+ predictors and no correction for multiplicity, if you don't find at least a smattering of statistically significant violations of the parallel-odds assumption, then you're doing something wrong.

[Xue Gao]

When I read https://www3.nd.edu/~rwilliam/xsoc73994/oglm01.pdf by Richard Williams, I am thinking that is it heterokedasticity that results in the difference between brant and omodel.
The appearance of heteroskedasticity is actually caused by other problems in model specification, e.g. variables are omitted, variables should be transformed (e.g. logged), squared terms should be added.
So I think if there is a problem in model specification. I take the logarithm of income and test the parallel line assumption. After that the results of tesing parallel line are similar between brant command and the omodel command.
However, if I take the logarithm of income in models, how to interpret the meaning of ln(income) coefficient. The one unit change of ln(income) is that income become 2.718 times?
And how to complete the adjusted predictions and marginal effects of income? Is it not appropriate to directly calculate the adjusted predictiona and marginal effects of ln(income)?
Thank you very much!

[Richard Williams]

The empirical results presented are all consistent with sticking with ologit. I don’t see why you think there is a conflict. Taking the logarithm of income may be a good thing to do, but it seems like a separate issue from the discussion of parallel lines.