Violated Wald test for gologit2 (generalised ordered logistic regression)

[Amina Keygan]

Hi all,

I am examining the relationship between intended family size (dependent ordinal variable 0-4+) and numerous explanatory variables (education, income etc). I have run ordered logistic but the models are in violation of the Brant test.
Following I conducted a generalised ordered logistic (gologit2) but the model appears to still be failing the Wald test. I have included the output for the Wald test below. Given these results, is it recommended I know move to a multinomial logistic regression?
Any help greatly appreciated.
Amina.


STATA output:

Testing parallel-lines assumption using the .05 level of significance...

Step 1: Constraints for parallel lines imposed for engborn (P Value = 0.6975)
Step 2: Constraints for parallel lines imposed for nosib (P Value = 0.3770)
Step 3: Constraints for parallel lines imposed for sib3 (P Value = 0.5682)
Step 4: Constraints for parallel lines imposed for age45 (P Value = 0.1917)
Step 5: Constraints for parallel lines imposed for age35 (P Value = 0.2861)
Step 6: Constraints for parallel lines imposed for certdip (P Value = 0.1185)
Step 7: Constraints for parallel lines imposed for ozborn (P Value = 0.1910)
Step 8: Constraints for parallel lines imposed for ptemploy (P Value = 0.1872)
Step 9: Constraints for parallel lines imposed for secedu (P Value = 0.0645)
Step 10: Constraints for parallel lines imposed for age25 (P Value = 0.1062)
Step 11: Constraints for parallel lines imposed for age40 (P Value = 0.0793)
Step 12: Constraints for parallel lines imposed for age30 (P Value = 0.0807)
Step 13: Constraints for parallel lines are not imposed for
unemploy (P Value = 0.00579)
married (P Value = 0.00387)
defacto (P Value = 0.00004)
sib1 (P Value = 0.00007)
sib4 (P Value = 0.01390)

Wald test of parallel-lines assumption for the final model:

( 1) [0]engborn - [1]engborn = 0
( 2) [0]nosib - [1]nosib = 0
( 3) [0]sib3 - [1]sib3 = 0
( 4) [0]age45 - [1]age45 = 0
( 5) [0]age35 - [1]age35 = 0
( 6) [0]certdip - [1]certdip = 0
( 7) [0]ozborn - [1]ozborn = 0
( 8) [0]ptemploy - [1]ptemploy = 0
( 9) [0]secedu - [1]secedu = 0
(10) [0]age25 - [1]age25 = 0
(11) [0]age40 - [1]age40 = 0
(12) [0]age30 - [1]age30 = 0
(13) [0]engborn - [2]engborn = 0
(14) [0]nosib - [2]nosib = 0
(15) [0]sib3 - [2]sib3 = 0
(16) [0]age45 - [2]age45 = 0
(17) [0]age35 - [2]age35 = 0
(18) [0]certdip - [2]certdip = 0
(19) [0]ozborn - [2]ozborn = 0
(20) [0]ptemploy - [2]ptemploy = 0
(21) [0]secedu - [2]secedu = 0
(22) [0]age25 - [2]age25 = 0
(23) [0]age40 - [2]age40 = 0
(24) [0]age30 - [2]age30 = 0

chi2( 24) = 42.15
Prob > chi2 = 0.0124

An insignificant test statistic indicates that the final model
does not violate the proportional odds/parallel-lines assumption

[Maarten Buis]

First, you need to tell us where you got gologit2 from. It is user written software, and there are usually multiple versions floating around on cyber space. If you do not tell us where you found gologit2 we could be talking about different versions, which would just waste everybodies time.

Second, performed a test using a gologit2 model of a ologit model, so you should get (asymtotically) the same results as brant (also user written, so the first comment also applies). But, that does not tell you anything about the gologit2 model, it only tells you something about an ologit model.

Third, your test tells us that you can reject the parallel lines assumption. This does not tell us anything new: a model is supposed to be a simplification of reality, and simplifaction is just another word for "wrong in some useful way", so we would expect and want that assumption to be wrong, we just don't want it to be too wrong. This decision we can only make by looking at the coefficients of the unconstrained model and see if you think that the deviations are "too large". The problem is that humans are very "good" at seeing patterns in random data. This is where the statistical test comes in: It protects you from seeing a pattern where none exist. So, if you get a significant result you can look at the coeffcients and see if you think that there is a problem with the ordered logit model. If you did not get a significant result, the potential patterns you would see in parameters could be either real or just random noise, and there would be no way to differentiate between these. In that case you would have to look outside the data in order to make a decision of whether or not the ordered logit model is appropriate (e.g. other studies that used similar data).

[Brendan Halpin]

As Maarten says, models are always imperfect, and if you have a large data set, you have enough information to show that it is wrong.

However, I'd like to add the point that there is no advantage in switching to multinomial logit: an unconstrained gologit2 model is essentially a different parameterisation of a multinomial model, and the mlogit model will have the same misfit. Rather, I'd be inclined to simplify the gologit2 model by seeing which, if any, variables do not violate the proportional-odds/parallel-lines assumption (use the option autofit).

[Maarten Buis]

The main point I made was that the test Amina performed did not test the gologit2 model. In fact, we know that the parallel lines assumption was not violated even without testing since a completely general gologit2 model does not impose the parallel lines assumption.

An interesting alternative to the gologit2 model would be a sequential logit model. That would model the probability of getting the first child, given that one has one child the probability of getting a second child, given that one has 2 children the probability of getting a third child, etc. If we ignore twins, that gets close to how these events actually occured.

[Amina Keygan]

Thank you for your comments.

Maarten- the Brant test and gologit2 programs were both installed through stata12. I am modelling family size intentions of childless individuals so parity is not a factor in this model (well I suppose it is but it is held constant at zero for all). Given this, would you still suggest that a sequential logit model might be useful?

Brendan- the code used for the gologit2 was as follows:
gologit2 famsize age25 ......., autofit lrforce or

Any other advice or suggestions would be greatly appreciated.
Amina.

[Richard Williams]

This occasionally happens, which is part of the reason I added that test after running autofit. Supposedly, after autofit has been run, all variables that violate the proportional odds assumption have had the assumption relaxed, and hence you would expect that final test to show that no variables now violate the assumption. But instead, the test is coming up significant. It is like when the global F for a model is significant but none of the individual T values are. The F test implies that at least one variable has an effect that differs from zero, but the T tests don't indicate what that variable is. Similarly, the gologit2 test is implying that at least one variable still violates proportional odds, but you can't tell what it is.

This seems like an awful lot of variables, and you don't indicate how large your N is. Maybe you are trying to do too much. See the gologit2 troubleshooting FAQ, and pay particular attention to the suggestions about simplifying the model. http://www3.nd.edu/~rwilliam/gologit2/tsfaq.html Or, consider Maarten's advice and see if some other type of model works better.

[Amina Keygan]

Richard,

Thank you for your response. Very much appreciated.

Apologies, I am running the models separately for men and women. N (men)= 2044 and N (women)= 1715.

Will, as you suggest, in FAQ section, add variables gradually and/or combine variables (particularly the age variables, perhaps into 10 year groupings instead of 5).

Many thanks,
Amina.

[Maarten Buis]

I did not ask which version of Stata you were running, though you should also have told us that since you are running an older version. I asked you where you found brant and gologit2, as there can be different versions of these programs as well.

For intended family size I would not use a sequential logit model. I would probably end up using an mlogit. There is ordering in the categories, but it is probably interesting to see how one variable works out differently for the comparison 0 versus 1 child and 0 versus 4+ children. Most models for ordered data gain parsimony by putting constraints on such effects, which can fine and very useful, but if you are interested in differences in these effects across outcome categories than that is not what you want.

[Amina Keygan]

Thank you Maarten. Very helpful.