Producing ROC curve using repeated measures

[sandra_a]

Hi,
I have a dataset of patients who were given a diagnostic test repeatedly to see the changes in fluid response. So, patients were administered fluids three times at different time interval. Then their change in body fluid was monitored and recorded. There is a diagnostic test which measures positive fluid change. I want to find out how accurate the diagnostic test is.

I need to produce ROC curves taking into account each patient was given repeated measures (taking into account variation within patient as well as variation between patients). There is great variation within a patient, hence taking an average/median/max/min wouldn't work well. How do I go about producing ROC curves with repeated measures in STATA?

Hope the question is making sense. TIA for your help.

Thanks
Sandra

[Joseph Coveney]

"Then their change in body fluid was monitored and recorded." Is that what you're taking to be the reference diagnosis (truth) against which to compare the diagnostic test's result? If so, it seems to be a continuous measurement, and not a disease/no-disease kind of reference diagnosis that ROC curves are based upon. In other words, it's not clear from your description how you would produce an ROC curve even if there were no repeated measurements within patients.

[sandra_a]

Sorry for not explaining it properly. There is a pre fluid level taken prior to the administration of the fluid challenge. There is a post fluid level taken. The changes in before and after the fluid challenge is monitored and calculated. If a patient's fluid change increases by more than 10%, then it's a positive response (i.e. patient needed the fluid).

There is another variable which is said to be associated to the fluid response(it's a continuous variable). I want to see at which threshold the variable is associated with a positive fluid response. Hope that makes sense.

[Steve Samuels]

Sandra,
Not using your full name (here not signing with your last last name as well as first) is a violation of Statalist protocol-please see http://www.statalist.org/forums/help#gfaq_postingadvice.

[sandra_a]

Dear Steve,

That's the username I chose. Sorry, I wasn't aware of the protocol. I just joined the forum. I have absolutely no issue with displaying my full name. I can't seem to find the option to display my full name. Please tell me I can change my username without having to register again. Thank you.

[Nick Cox]

You can. Contact the administrators using the "Contact us" button on the home page.

[Joseph Coveney]

If I've read their paper correctly, Liu and Wu advocate fitting a generalized linear mixed model to the repeated measurements, making predictions from the fitted model, and then computing the area under the curve (AUC) of the receiver operating characteristic (ROC) function in a conventional manner from the observed values for the reference diagnoses and corresponding predictions.

A little simulation (see attached log file and graph) indicates that you get a boost in estimated AUC when you use all of the available data from a repeated-measurements set-up and the intraclass correlation is moderately high (in this case, 70%). This boost above that for a randomly chosen cross-sectional ROC curve from each participant is most evident when the ROC AUCs are in the low range (less than 75 or 80%), and diminishes as the ceiling is approached. The direction of truth is actually got wrong (ROC AUC less than 50% in the graph) on occasion, too, in the low range when a single point is randomly chosen from each participant for construction of the cross-sectional ROC curves. In the simulation, use of all four time-points' data for each participant completely avoids this sign error.

[Roger Newson]

I would use the -somersd- package, which you can download from SSC, and which has the option -transf(c)- to get Harrell's c (also known as the ROC area). The -somersd- has a -cluster()- option for clustered data, such as repeated measurements. And you can use the option -funtype(vonmises)- if you want the full ROC area, including comparisons within subjects.

I hope this helps.

Best wishes

Roger