Interpretating classification of logistic model

[Sharon Rith]

After using 'estat classification' for the logit model I am unsure how to interpret the output (see below)
My model correctly classifies 78% of cases but has relatively low specificity.
Does this output somehow reinforce the robustness of my model or can I interpret these classification rates in any useful way regarding my specification?

[Clyde Schechter]

The overall percentage of cases classified correctly is generally not a very useful statistic. It is sensitive to the distribution of the "true" values in the data, which can vary quite widely from one situation to the next. The sensitivity and specificity are the more important statistics. I don't get why you consider a specificity of greater than 98% to be low--that's outstanding! On the other hand your sensitivity at just under 12% is not impressive. Remember, though that this output was generated using a predicted probability of 0.5 as the cutoff (the default for -estat classification-). With different cutoffs (which can be specified in the -cutoff()- option)*, you can trade off specificity to get more sensitivity. It may be that some other cutoff will yield a combination of sensitivity and specificity that is on balance better. Better, in this context, depends on your situation. Low sensitivity means that you are missing cases, whereas low specificity means you are making false-positive classifications. The consequences of those two errors are, typically, quite different, and the best combination of sensitivity and specificity will depend on just how much importance you attach to each. To do this formally, you need to specify a loss function that tallies up the disutility of missing cases and the disutility of false-positive classifications. Then you want to find the cutoff value of predicted probability that minimizes the loss function given the distribution of cases and non-cases in the population of interest.

That said, people often have difficulty with judging these disutilities, and are often uncomfortable with the very idea of making a value judgment. For that reason, various "judgment-free"* indices of test performance are commonly seen in the literature. One is the Youden index, which is just sensitivity + specificity (or, some people use that minus 1). Again, you need to check this at different cutoffs and then find the one that does best. Another is the area under the ROC curve (see -lroc- or -roctab-, among others). The area under the ROC curve is a single statistic that summarizes the ability of the test to distinguish cases from non-cases and it inherently takes into account all possible sensitivity-specificity tradeoffs. A full explanation is too long to give in a Forum post. The Wikipedia page on Receiver Operating Characteristics gives a reasonable overview.

*I put judgment-free in scare-quotes because, in fact, using the Youden index to evaluate the quality of a text simply hides the fact that you have implicitly chosen to count false-positive and false-negative results as equally harmful, without coming out and saying so.

As an aside, for the future please do not attach screen shots to present Stata output. Yours is just barely legible on my computer; many are not readable at all. Even when they are readable, to view them entails obscuring the original post, and going back and forth between the screen shot and the original post is cumbersome. Why put obstacles in the path of people who are trying to help you? The best way to show Stata output is to copy from the Results window or your log file and paste into a code block here on the Forum. (If you don't know how to set up a code block, see FAQ #12 for instructions.)

[Sharon Rith]

Thanks very much Clyde! Sorry about that, I obviously made a typo for which was relatively low and meant sensitivity! I did not know about the screenshot problem, I shall endeavour to use the code block from now on. I shall consider the ROC curve analysis, sounds more useful than the classification statistic as you say. Thanks a lot.