P-value for AUC after roctab command

[Dusan Petrovic]

Dear Statalisters,

I am performing rocregression and I would like to know whether it is possible to compute the p-value for Area Under Curve?

My rocreg command is the following :

roctab categorical_var continuous_var if index==1

This command provides an output with SE and 95%CI for AUC, but I would like to know whether it is possible to extract a p-value for a test testing whether AUC is significantly different from 0.5.

(Perhaps with the test command).


Thank you very much for your kind help

Best

Dusan

[Clyde Schechter]

Assuming a normal distribution for the sampling error of the AUC, running the following code immediately after the -roctab- command would do it

local z = (r(area)-0.5)/r(se)
display "p = " %05.3f =2*normal(-abs(`z'))

Note: This is a two-tailed p-value. If your study design is such that you completely exclude the possibility of an inverse association between predictor and reference criterion, then a 1-tailed test might be more appropriate. If so, just eliminate the "2*" from the -display- command.

That said, I have rarely if ever seen anybody do this kind of hypothesis test. At least in epidemiologic and clinical research with ROC curves it is far more common to just report the AUC and its confidence interval. The null hypothesis of AUC = 0.5 isn't really very interesting (and a priori has no plausibility in any realistic application). If you really want to just test whether your predictor has any association with the reference criterion, it would be more conventional, I think, to just do a logistic regression.

[Dusan Petrovic]

Hello Clyde,

Thank you very much for your answer and advices. I will discuss with my supervisor.

Best

Dusan