Reporting predicted probabilities

[Jose Navarro]

Hello. I've been asked to report the results of a logit model (binomial logistic regression) in predicted probabilities instead of coeficients. I'm using margins (atmeans) to compute the results. My independent variables are all categorical.

My question is regarding the p value that I should report (the "stars"). In the results of the logistic regresion I can only see the results of the categories that are not the reference category, whereas when I compute the predicted probabilities with margins I obtain a probability value for all the categories, including the reference category. I'm a little confused with this.

Thank you

[Clyde Schechter]

It is probably best to just ignore the p-values in your -margins- output. They are tests of the null hypothesis that the probability for that subset is zero. Of all the implausible straw-men null hypotheses in the universe, this is about as bad as you can get! These p-values are truly meaningless. In fact, not only are these null hypotheses not plausible, they are essentially impossible: if the probability of your outcome in a certain group were really zero, you would not observe those outcomes at all in that group, and that indicator variable would be omitted from the logistic regression due to perfect prediction. It is better to report the confidence intervals, which give a sense of the precision with which your predicted probabilities have been estimated. Even those can be confusing, because if the predicted probability is close to zero or one, one of the confidence limits may go beyond the 0-1 range of admissible values.

When you use -margins- to calculate marginal effects, rather than predicted probabilities, then it may make sense to report p-values. Those p-values make as much sense (no more, no less) as the p-values of the coefficients. And the null hypothesis of a marginal effect being zero, though probably not plausible when taken literally in most research contexts, is at least potentially true or fairly close to true, and is of some actual interest as a "benchmark" to compare the observed results to.

[Bruce Weaver]

Adding to what Clyde said (in #2), I think it is far more conventional in many fields to report differences in log-odds, or equivalently odds ratios, when reporting marginal effects for a logit model. (Include the predict(xb) option for margins to get results on the log-odds scale rather than as predicted probabilities.) HTH.