Average Marginal Effects and logit statistics

[Ava Cerrid]

Dear everyone,

I am working for the first time with average marginal effects (AME) and have some questions regarding this:

1) In order to obtain the AME I used the "logit" command with all my variables and then used the " margins, dydx(*)" command. My question is now whether I can also interpret the LR chi2(21) , the Prob > chi2 and the Pseudo R2 statistics of the former calculated logit model, or do they exclusively belong to the logit model? In my understanding this should not be possible, but my superviser still said that I should interpret these statistics.
2) I also used the "estat ic" command to get the AIC and BIC statistics. Now again, are these statistics only refering to the logit model or also to the AME model?

If this is also interesting for you guys to know: I am doing a binary logistic regression with predictor and control variables of metric, ordinal and nominal scale.

Thank you very much!

[Richard Williams]

Welcome to Statalist.

I don't think of these as two different models. You can discuss the stats you get from both the logit command and the subsequent margins command. Using one does not preclude use of the other.

[Ava Cerrid]

Thank you very much for the quick response and the welcoming!

I am very sorry but I guess I was quite imprecise with my question. I calculated 2 different models (Model 1 with only predictors and Model 2 with predictors and control variables). I do know that if I use the logit command and afterwards the margins command, that these two ouputs are linked. Thus, regarding the AIC and BIC I am not asking whether it is possible to compare the logit and the AME model, but if it is ok to use the AIC and BIC to compare Model 1 and Model 2.
So basically what I want to know is if I can use the AIC, BIC, chi2 and pseudoR statistics, even if I don't report the results of the logit model (what I am going to do, since I will just focus on the AMEs) or whether these statistics are only to be interpreted if I also report the logit results since they are linked to them. If this is true, I could only report the results of the AME output.

I hope this is more clear. I am sorry for the misunderstanding..

[Jordan Sydenham]

Following on from what Richard said, the Average marginal effect is calculated using the coefficients from the Maximum likelihood estimation of your Logit regression, it is not a separate model, it is rather another useful statistic that can be calculated from the model (just like the Pseduo R2 for example) . The AME's are used in the case of a logit model because it is by nature a non-linear model meaning the coefficients are difficult to interpret on their own (one can only really use them to determine the direction on the effect and not the magnitude). The Prob > chi2 and the Pseudo R2 statistics calculated from the model are therefor still relevant to your discussion. As would be the AIC and BIC statistic.

[Jordan Sydenham]

You cannot compare AIC or BIC from different models. You can compare pseudoR statistics from diffrent logit models.

[Ava Cerrid]

Hello Jordan,

thank you for your answer! Isn't the purpose of the AIC and BIC that I can compare different models? E.g. If my BIC in Model 1 is lower compared to Model 2, the fit in Model 1 because the lower the AIC and BIC the better? Why else would I compute the AIC and BIC then, because they have nor real explanatory power alone, or am I mistaken?
The answer of your previous post regarding chi2 and pseudoR is very plausible, thanks!

[Jordan Sydenham]

AIC and BIC are normally used to compare models with the same specification just a different number of lags. If you are including different variables then the use of AIC or BIC is no longer appropriate, at least that is what I was taught during my undergraduate studies.

[Jordan Sydenham]

To add to this, in terms of a logit model the AIC and BIC is used to compare with a Probit model that includes the exact same specification.

[Ava Cerrid]

Ok I see. We usually introduce the control variable stepwise and I learned that the use of AIC and BIC, even if the set of included variables differ, is ok. I think I need to do more research regarding that problem.

[Richard Williams]

You can use AIC and BIC to compare models, especially if they are nested. Be careful that the same cases are being analyzed though, since as you add variables to a model missing data can cause some cases to drop out. For example, it is legit to do

webuse nhanes2f, clear
logit diabetes i.sex, nolog
est store m1
logit diabetes i.sex weight height, nolog
est store m2
lrtest m1 m2, stats

. lrtest m1 m2, stats

Likelihood-ratio test                                 LR chi2(2)  =    132.23
(Assumption: m1 nested in m2)                         Prob > chi2 =    0.0000

Akaike's information criterion and Bayesian information criterion

-----------------------------------------------------------------------------
       Model |          N   ll(null)  ll(model)      df        AIC        BIC
-------------+---------------------------------------------------------------
          m1 |     10,335  -1999.067  -1997.369       2   3998.738   4013.224
          m2 |     10,335  -1999.067  -1931.255       4    3870.51   3899.483
-----------------------------------------------------------------------------
Note: BIC uses N = number of observations. See [R] BIC note.

In this case, whether you use an LR test, BIC, or AIC, you wind up preferring the model with the added variables.

For more, see

https://www3.nd.edu/~rwilliam/xsoc73994/L05.pdf

[Richard Williams]

As far as analyzing the same cases throughout, a few techniques are described on p. 3 of

https://www3.nd.edu/~rwilliam/xsoc73994/MD01.pdf

[Ava Cerrid]

Thanks again Richard. I knew about that issue before and all of my models have the same sample size!
I really value the quick response and the example.