Model selection using AIC and BIC criterion

[Adamu Idris]

Dear respected members,

Can anyone assist me to solve my problem with regards to model selection in logistic regression? when AIC of a model (for example model 1) consisting of all independent variables is smaller and its BIC is large compared to the second model (for example model 2) that has few independent variables and vice versa. Which model among the two should I select? Your suggestions are important. Below is the output for the two models and I used 'estat ic' after xtlogit command to obtain the output.


Akaike's information criterion and Bayesian information criterion
MODEL 1
----------------------------------------------------------------------------------------------------
Model | Obs ll(null) ll(model) df AIC BIC
-------------+------------------------------------------------------------------------------------
. | 596 . -151.4472 20 342.8944 430.6992
-----------------------------------------------------------------------------------------------------

Akaike's information criterion and Bayesian information criterion

MODEL 2
-------------------------------------------------------------------------------------------------
Model | Obs ll(null) ll(model) df AIC BIC
-------------+---------------------------------------------------------------------------------
. | 596 . -158.9105 15 347.821 413.6746
------------------------------------------------------------------------------------------------

Thank you

Adamu Idris

[Richard Williams]

First off, your output would be easier to read if you used code tags. See pt. 12 of the FAQ.

The chi-square contrast between the models can also be used.

You don't have to choose based on BIC or AIC. Especially given the mixed results, you can go with whatever model you think is more plausible. Also, there may be some intermediate model, with more variables but not all, that would provide the best fit.

But just in general, don't think raw empiricism has to decide what model you go with. Indeed, you run the risk of getting deceptive results if you run a bunch of models and then choose the best fitting one. P values can be wrong in such situations.

[Adamu Idris]

Thank you professor for the remark as well as the observation.