Multinominal model - command for % correctly predicted

[Eileen Mitchell]

Hello, I am running a multinomial model and want to check what % of the entire model was correctly predicted.

I want to use a command similar to estat classification however this type of command does not work on the multinomial model.

logit $ylist $xlist
estat classification
output reads: correctly classified x% etc.

mlogit $ylist $xlist
estat classification
output reads: invalid subcommand classification

I have tried:
* Multinomial logit marginal effects*
margins, dydx(*) atmeans predict(pr outcome(0))
margins, dydx(*) atmeans predict(pr outcome(1))
margins, dydx(*) atmeans predict(pr outcome(2))

* Multinomial logit predicted probabilities
predict pmlogit0 pmlogit1 pmlogit2, pr
summarize pmlogit0 pmlogit1 pmlogit2
tabulate $ylist

Output:
pmlogit0 = mean = 0.50
pmlogit1 = 0.20
pmlogit2 = 0.298

My original results where
0 = 49.81 % selected this
1 =19.88 % selected this
2 = 30.31 % selected this

However I want to run to see the overall fit of the model.

Would I be correct to say that the % correctly predicted is 98%?
0.5+0.2+0.298 =0998
49.81+19.88+30.31=100 (100-99.8=0.2) = ( 100% - 2% =98)? : I suspect this is not correct, but thought I would check.

[Mike Lacy]

There are a lot of things about your question I don't understand. For example, in the material you have shown, I don't know why you would want to obtain the marginal effects, or to summarize the predicted probabilities, since neither of those speaks to your desire to compare actual and predicted categories. So, I may be misunderstanding what you want.

However, presuming that part of what you want is to compare the predicted to the actual response category, here's something to get you started,

mlogit y .....
predict  pr0 pr1 pr2, pr
// For each observation, find category with maximum predicted probability, and use that as
// the predicted category value.
gen byte predcat = .
gen maxp = -1
forval i = 0/2 {
  replace predcat = `i' if (pr`i' > maxp)
  replace maxp = pr`i' if (pr`i' > maxp)
}
// Compare observed and predicted.
tabulate y predcat

From the tabulation, you could then by hand or program calculate the various classification statistics that are relevant to your problem.

[Weiwen Ng]

This actually gets at something covered in this thread on modal class probability after a finite mixture model, and it's also relevant in logistic regression. Basically, after multinomial logit, you have the model-predicted probability that each person was a member of each class. The model does not predict what class the person belongs to. You can choose to apply a definition of what "belongs to" is, and as Mike said, the observation's modal class is probably an excellent definition.

I think that your original code and math were getting at something like how closely the model-predicted probabilities correspond to the observed ones. I am not aware of any statistics that are based on that definition and that are used to test goodness of fit for multinomial logit. If you are getting a PhD in statistics, then maybe this is something to investigate! You can compare different nested multinomial logit models to see which fit the data better using the likelihood ratio test, but that is only a relative indicator (NB, you can use that test on many types of models, not just multinomial logit).

[Eileen Mitchell]

Thanks all, the code you provided Mike appears to work thanks - Yes - I wanted to compare the predicted to the actual response category. Many thanks for all your help.