convergent validity

[Nina Palker]

How would I calculate the convergent validity of two diagnostic tests where both outcome measures are binary?

[Weiwen Ng]

For others' information, convergent validity is a term used in psychometrics. Basically, two tests measure similar latent constructs (e.g. depressive symptoms and negative mood). They should be correlated. With tests that have sum scores, you can simply show the correlation of the sum scores. I believe this concept would generalize to two different tests that measure the same latent construct.

Of course, Nina noted that she has two binary tests. As a side note, if they're binary because they were continuous measures but someone dichotomized them, I think most people would recommend getting the original continuous scores if you can.

If you can't or if the tests are inherently binary (e.g. physician diagnosis of something), then I haven't seen this situation very much, so this recommendation is based on intuition. If the tests are for the same latent construct and you have one gold standard measure, you could show a cross tabulation of test A vs test B, plus kappa (i.e. show that the tests have substantial agreement with each other), and also show how much each test relates to your gold standard.

If you don't have a gold standard, then you're stuck with showing how much the tests agree (i.e. two way table and show the value of kappa).

If the tests measure two similar but not identical latent constructs, then I'm a bit stumped. I'd still wager that kappa is informative.

[Nina Palker]

Thanks Weiwen, this is very helpful