Combining ROC curves using roctab

[Anna Tavakkoli]

Hi,

I appreciate anyone who can help me with this problem. I am trying to combine two ROC curves into the same graph. I am trying to validate the definition of a failed procedure using two different published definitions. I didn't include data ex since it is just 0's and 1's.

Variables:
Failed_gs: gold standard definition for failed procedure, coded 0,1
C: new definition #1 for failed procedure, coded 0,1
i: new definition #2 for failed procedure, coded 0,1

Does anyone have any guidance on how I can combine these two ROC curves into one graph? Thank you

 roctab failed_gs c, detail

Detailed report of sensitivity and specificity
------------------------------------------------------------------------------
                                           Correctly
Cutpoint      Sensitivity   Specificity   Classified          LR+          LR-
------------------------------------------------------------------------------
( >= 0 )          100.00%         0.00%       83.61%       1.0000     
( >= 1 )           69.61%        85.00%       72.13%       4.6405       0.3576
( >  1 )            0.00%       100.00%       16.39%                    1.0000
------------------------------------------------------------------------------


                      ROC                    -Asymptotic Normal--
           Obs       Area     Std. Err.      [95% Conf. Interval]
     ------------------------------------------------------------
           122     0.7730       0.0469        0.68108     0.86500

roctab failed_gs c, table graph summary 

roctab failed_gs i, detail
Detailed report of sensitivity and specificity
------------------------------------------------------------------------------
                                           Correctly
Cutpoint      Sensitivity   Specificity   Classified          LR+          LR-
------------------------------------------------------------------------------
( >= 0 )          100.00%         0.00%       83.19%       1.0000     
( >= 1 )           94.95%        60.00%       89.08%       2.3737       0.0842
( >  1 )            0.00%       100.00%       16.81%                    1.0000
------------------------------------------------------------------------------


                      ROC                    -Asymptotic Normal--
           Obs       Area     Std. Err.      [95% Conf. Interval]
     ------------------------------------------------------------
           119     0.7747       0.0573        0.66249     0.88700

roctab failed_gs i, detail

[Leonardo Guizzetti]

Since both tested variables (i and C) are binary, each ROC curve is defined by precisely 1 point (the condition where the cutpoint is >=1 in this case).

Here's an example of one way to do it. You may adapt it for your own case, or again, straightforwardly plot the few points necessary.

clear *
cls

webuse hanley

rename rating rating1
gen rating2 = min(max(rating + round(runiform(0, 2)), 1), 5)

// Get Se and Sp of Rating 1
qui roctab disease rating1 , nograph detail
mat list r(detail)
mat Roc1 = r(detail)
svmat Roc1, name(roc1_)
rename (roc1_2-roc1_3) (roc1_se roc1_sp)
drop roc1_?

// Get Se and Sp of Rating 2
qui roctab disease rating2 , nograph detail
mat list r(detail)
mat Roc2 = r(detail)
svmat Roc2, name(roc2_)
rename (roc2_2-roc2_3) (roc2_se roc2_sp)
drop roc2_?

foreach v of varlist roc1* roc2* {
  replace `v' = `v' / 100
}
gen double roc1_1minus_sp = 1 - roc1_sp
gen double roc2_1minus_sp = 1 - roc2_sp

twoway (function y = x, range(0 1)) || ///
       (sc roc1_se roc1_1minus_sp, c(l)) || ///
       (sc roc2_se roc2_1minus_sp, c(l))

[Anna Tavakkoli]

Hi,

Thank you so much ! This worked but I had to modify the code and my graph doesn't look the same. When you rename rating rating1 and then use rating in the next line I get an error message there so I had to substitute rating2 with the variable name of the second test. Is there any way to remedy this? Mainly it is because your graph looks so much nicer.

rename c c1
gen rating2 = min(max(c1 + round(runiform(0, 2)), 1), 5)

// Get Se and Sp of Rating 1
qui roctab failed_gs c1 , nograph detail
mat list r(detail)
mat Roc1 = r(detail)
svmat Roc1, name(roc1_)
rename (roc1_2-roc1_3) (roc1_se roc1_sp)
drop roc1_?

// Get Se and Sp of Rating 2
qui roctab failed_gs inhospital , nograph detail
mat list r(detail)
mat Roc2 = r(detail)
svmat Roc2, name(roc2_)
rename (roc2_2-roc2_3) (roc2_se roc2_sp)
drop roc2_?

foreach v of varlist roc1* roc2* {
  replace `v' = `v' / 100
}
gen double roc1_1minus_sp = 1 - roc1_sp
gen double roc2_1minus_sp = 1 - roc2_sp

twoway (function y = x, range(0 1)) || ///
       (sc roc1_se roc1_1minus_sp, c(l)) || ///
       (sc roc2_se roc2_1minus_sp, c(l))

[Leonardo Guizzetti]

What is the error you get, specifically?

Your graph will not look the same because there is only one effective cut point, whereas there are 4 in my example. Your graph will look like two overlapping triangles.

[Anna Tavakkoli]

Yes you are right! They are two overlapping triangles! Perfect thank you so much. I really appreciate it