Possible to do a meta-analysis on Cohen's Kappa Interrater agreement?

[Heather Thomas]

Hi,
I am a beginner at Stata and am attempting to do a meta-analysis on a number of studies reporting results as Cohen's Kappa Interrater agreement, with 95% CI.

I have previously utilized OR (using metan) and percentages (using metaprop) with no difficulties. However, am struggling to determine what I should be using for Kappa agreement.

Any advice would be greatly appreciated.
Thank you!

(Using Stata/IC 15.0 for Windows)

[Bruce Weaver]

Hello Heather. Have you looked at metaan?

• http://www.stata-journal.com/article...article=st0201
• https://www.researchgate.net/publica..._meta-analysis

From the help:

Syntax

    metaan varname1 varname2 [if] [in] [, options]

    where

      varvame1 the study effect sizes.

      varvame2 the study effect variation, with standard error used as default.

Type findit metaan to find and install it.

HTH.

[David Fisher]

Hi Heather,

I'm not sure that Bruce's post really gets to the nub of the problem. There are many meta-analysis commands that take in effect sizes and standard errors or variations or confidence intervals and produce a result. You yourself mentioned metan, which is absolutely fine. metaan is simply an alternative (as is admetan/ipdmetan, my own meta-analysis package).

Your problem, as I understand it, is how to back-calculate standard errors for Kappa statistics from the confidence intervals you have available. In particular, whether a transformation exists of Kappa and it's confidence interval under which we can use a Normal approximation. For example: if you have an Odds Ratio and a confidence interval, then you would log-transform those three values, and then estimate the standard error of log(OR) as [log(UCI) -- log(LCI)]/(2*1.96). metan will do this calculation for you if you feed it the logOR and log confidence limits. This is important, since standard inverse-variance meta-analysis relies on a Normal approximation to produce reliable pooled results.

Unfortunately, I'm not knowledgeable enough on Kappa statistics to be able to help. A quick read through the Stata manual entry for kappa suggests that standard errors are not straightforward to calculate. You may need to find other meta-analyses of such statistics and read through their methods sections for suggestions on how to proceed.

Best wishes,

David.