BIC & AIC_choose best model

[Hui SHI]

Hi,

I am confused with Bayesian information criterion (BIC) and Aikaike's information criterion (AIC) when they are negative values.

When they are positive values(+), the model with lower BIC and AIC values is better, Is it right？

When they are negative values(-), is it still think the model with lower BIC and AIC values is better, in this condition, which mean that the bigger absolute value of BIC and AIC are better when they are negative values.


I hope you can confirm this question for me.

Thanks. very much.

[Roman Mostazir]

Closer they are to zero, better the model. Say, for example, if the log of the likelihood (ll) values are presented negative and have the values -3 and -4, the model with ll= -3 is closer to zero than the model with ll=-4. Some software present the positive values by multiplying the ll values with -2. In this case, -3*-2 = 6 and -4*-2 = 8 and again 6 is closer to zero than 8. Either way, ll-closer to zero is the preferred model.

[Hui SHI]

Closer they are to zero, better the model. Say, for example, if the log of the likelihood (ll) values are presented negative and have the values -3 and -4, the model with ll= -3 is closer to zero than the model with ll=-4. Some software present the positive values by multiplying the ll values with -2. In this case, -3*-2 = 6 and -4*-2 = 8 and again 6 is closer to zero than 8. Either way, ll-closer to zero is the preferred model.

Thanks for your reply.

It makes sense to me now.

I have search on google: With negative values of AIC and BIC, comparing them to find the smallest values. This means- the more negative, the better.

We should compare them to compare which one close to 0.

Thanks very much.

[Sebastian Kripfganz]

Closer they are to zero, better the model. Say, for example, if the log of the likelihood (ll) values are presented negative and have the values -3 and -4, the model with ll= -3 is closer to zero than the model with ll=-4. Some software present the positive values by multiplying the ll values with -2. In this case, -3*-2 = 6 and -4*-2 = 8 and again 6 is closer to zero than 8. Either way, ll-closer to zero is the preferred model.

That's incorrect!

The model with the more negative value of the AIC or BIC is preferred, i.e. the model with a value further away from zero (if values are negative).

[Maxence Morlet]

In addition to Dr. Kripfganz's excellent advice, Hui SHI you may want to read this paper: Kuha, J. (2004). AIC and BIC: Comparisons of Assumptions and Performance. Sociological Methods & Research, 33(2), 188–229. https://doi.org/10.1177/0049124103262065

Generally, BIC is preferred to the AIC as it bears a component that accounts for sample size, unlike the AIC.

I couldn't name another paper right off the top of my head, but I'm quite sure Monte Carlo simulations have shown that the BIC performs better in small samples and may be preferable. Don't quote me on that though.

[Hui SHI]

That's incorrect!

The model with the more negative value of the AIC or BIC is preferred, i.e. the model with a value further away from zero (if values are negative).

Can I understand that the smaller absolute value of BIC and AIC, the model are better?

[Rich Goldstein]

no; -5 is smaller than -2 but 2 is smaller than 5

[Hui SHI]

no; -5 is smaller than -2 but 2 is smaller than 5

But in this case, 2 is much Closer to zero than 5

[Rich Goldstein]

so? read post #4 more carefully

[Hui SHI]

so? read post #4 more carefully

Thanks for reply. I copied the results of several model.

model 1: BIC= -6338.70
model 2: BIC= -6311.06
model 3: BIC= -6879.95

In this case, the best fitting model is model 2, because it is much closer to zero. Is it right?

[Hui SHI]

That's incorrect!

The model with the more negative value of the AIC or BIC is preferred, i.e. the model with a value further away from zero (if values are negative).

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

I give an example,

model 1: BIC= -6338.70
model 2: BIC= -6311.06------closer to zero
model 3: BIC= -6879.95------further away from zero, it is, the more negative

which model is best model if we only consider BIC.

Thanks

[Hui SHI]

In this table A1, authors choose one model with less negative value of BIC, -6311.47

[Hui SHI]

In this link, someone have asked the same question.

https://stats.stackexchange.com/ques...l-based-on-bic

1. ) -4.98749
2. ) -4.995782-----further away from zero, more negative
3. ) -4.9864------closer to zero

Note: based only on BIC.
In this example, the model with the smallest BIC value, -4.995782, can be taken as the preferred model;

This answer is different from the Table 1A.

[Hui SHI]

Can I understand that the smaller absolute value of BIC and AIC, the model are better?

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

In this article (Dynamics of Long-Term Patient-Reported Quality of Life and Health Behaviors After Adjuvant Breast Cancer Chemotherapy), authors chose one with smaller absolute value of BIC.


So I think if we based only on BIC, BIC avlue much closer to zero, that is mean, smaller absolute value of BIC, less negative, the model is better.


Is this conclusion right?

[Sebastian Kripfganz]

I do not know how the authors of these articles calculated the BIC. Without extra information, their choices seem wrong. When you base your decision on the AIC or BIC as reported by the official Stata command estat ic, you select the model with the smaller (i.e. more negative) value of those criteria.