Odd pooled estimate with metan

[Tom Yates]

I am using metan to display results from published studies alongside some similated results.

clear

input str15 Study str32 Estimate double(RR LL UL)
"Study 1" "Naive result, as published"  0.22 0.01 3.81
"Study 2" "Naive result, as published" 0.31 0.04 2.29
"Study 3" "Naive result, as published" 0.40 0.13 1.27
"Sim_Study 1" "Simulated, standard assumptions"  0.33 0.024 4.6
"Sim_Study 2" "Simulated, standard assumptions" 0.33 0.06 1.8
"Sim_Study 3" "Simulated, standard assumptions" 0.32 0.093 1.1
"Sim_Study 1" "Simulated, low assumptions"  0.49 0.044 5.5
"Sim_Study 2" "Simulated, low assumptions" 0.45 0.09 2.2
"Sim_Study 3" "Simulated, low assumptions" 0.47 0.15 1.4
"Sim_Study 1" "Simulated, high assumptions"  0.14 0.0048 4.4
"Sim_Study 2" "Simulated, high assumptions" 0.15 0.016 1.4
"Sim_Study 3" "Simulated, high assumptions" 0.36 0.038 3.4
"Study 4" "RCT" 0.42 0.05 3.28
end

gen ln_RR=ln(RR)
gen ln_LL=ln(LL)
gen ln_UL=ln(UL)

metan ln_RR ln_LL ln_UL, eform effect("RR") random null(1) lcols(Study) by(Estimate) nooverall

A minor point, but why does the confidence interval for the RCT subgroup differ subtly from that of the single trial?! Is it trying to force symetry lost due to a rounding error, when I inputted my simulated results?

Is there a way to get metan not to attempt to pool the result from a single study? I do want the pooled results from the other subgroups.

Thanks,
Tom

[David Fisher]

Hi Tom,

The discrepancy is due to an asymmetrical confidence interval for that study, as demonstrated by the following:

. disp 3.28/.42
7.8095238

. disp .42/.05
8.4

Standard meta-analysis models are based on inverse-variance weighting; so in order to proceed, metan needs to derive the standard error from the confidence interval, using the following formula (where the confidence limits UCI and LCI are on the interval scale; that is, logarithms of ratio measures):

StdErr = (UCI - LCI) / (2 * invnormal(.975))

To derive the pooled estimate, metan then re-derives the confidence limits based on this standard error. Therefore, the confidence limits will be perfectly symmetric around the point estimate.

Asymmetric confidence limits are a common problem in meta-analysis. Some packages/commands, such as Stata's built-in meta suite, will complain if they are insufficiently symmetric, forcing you to use the civartolerance(#) option.

There is currently no option to disable pooling of single-study subgroups in metan. Notice that metan will, at least, faithfully reproduce the inputted confidence limits for the study observation; no matter how asymmetric the limits. My first suggestion is to double-check your extracted data, just in case there is a typo. If not, then the following indirect approach will work:

. metan ln_RR ln_LL ln_UL, rr random lcols(Study) by(Estimate) nooverall nogr clear
. replace _EFFECT = _EFFECT[_n-1] if _USE==3 & _BY==5
. forestplot, useopts nostats nowt rcols(_EFFECT _WT)

This is saying, roughly: "first, please replace the pooled stats with the individual study stats for the fifth subgroup; then re-draw the forest plot without re-calculating anything".

I hope that helps.
BW, David.

[Tom Yates]

Thanks David, I suspected it was something like that. I am reliant on the published data, but that is the confidence interval that they report. I will apply your indirect approach.

With best wishes,
Tom