metan: A comprehensive update to the popular meta-analysis command

[David Fisher]

With thanks as ever to Kit Baum, I am excited to announce a major update to the user-written command "metan", version 4.0, now available via SSC.

Firstly, a bit of history: as described in this thread I previously released v3.x of the admetan / ipdmetan meta-analysis command suite, and presented it at the 2018 London UK Stata Conference. One of the discussions that emerged was the possibility of merging admetan with the ever-popular but venerable metan, which had not been updated for a decade and currently has no dedicated support. Over the past year I have worked with Professor Jonathan Sterne, a co-author of metan, and Professor Julian Higgins -- both eminent figures in the meta-analysis field -- to make this happen. An important aspect of this work has been to make the update as seamless as possible -- that is, to enable existing metan users to continue to use the command in the same way as before, if they so wish. We have also ensured that all default behaviour is in keeping with current best practice in the field.

Much of the "new" functionality is simply carried over from admetan, and as such is described in this thread. However, all new behaviour, and differences from the previous version and from Stata 16's meta suite, are fully documented in the *.sthlp files. In particular:

• Standard heterogeneity output, including I-squared, is always based on a single Q statistic calculated directly from the data (either Cochran's Q, or a common-effect alternative e.g. using the Mantel-Haenszel log-effect size and weights) rather than on tau-squared. Tau-squared is reported in a separate table. If the user also wishes to see I-squared as derived from tau-squared and sigma-squared (see Higgins & Thompson 2002), they may use the option isqparam (see documentation).
• Returned and saved effect sizes, standard errors and confidence intervals are *always* on the interval/log scale, even with Mantel-Haenszel methods. This avoids confusion when comparing between models (see below) and enables standardised interface with other commands. (However, note that previously-returned statistics such as r(RR) are still, additionally, available.)
• A major new feature: the results of multiple models -- inverse-variance, Mantel-Haenszel, DerSimonian-Laird, REML, and so on -- may be presented simultaneously beneath the list of individual studies on-screen and in a forest plot. Presented weights are taken from the first model in the list. All pooled results, heterogeneity statistics etc. are returned in matrices. This can also be combined with a (single) subgroup variable with by().
• metan now incorporates the functionality of the user-written programs metacum, metaninf and metaprop, respectively for cumulative and influence meta-analysis and meta-pooling of proportions. The older programs call metan from within them, and *should* still work OK (due to our desire for the update to be seamless), but it is recommended that metan now be used instead, with the options cumulative, influence or proportion as required (see documentation).

As a result of this update, the admetan package will no longer be maintained. However, ipdmetan remains as a separate package -- albeit with a dependency on metan -- and will continue to be maintained (a small update to ipdmetan is also now available via SSC).

Since this is such a major update, there are bound to be teething troubles; so please don't hesitate to contact me ([email protected]) with any bugs or other issues. If all else fails, the previous version of metan is still available within the package under the name metan9.ado.

I hope you find the updated package useful.

Season's greetings to all!

David.


David Fisher
Statistician
MRC Clinical Trials Unit at UCL, London, UK
e-mail: [email protected]


P.S. Apologies to anyone who has started threads here, or replied to existing threads, with issues related to admetan, ipdmetan or forestplot (or, indeed, metan) over the last couple of months; I've been putting all my energy into getting this update out. I will now try to catch up!

[Suhail Doi]

Hi David,

Thanks for this very useful package update. I tried the QE model on the data below:

* Example generated by -dataex-. To install: ssc install dataex
clear
input str23 study int n byte(a b c) int d float qi byte(q1 q2 q3 q4 q5) float(r1 r2 r3 r4 r5)
"Baronciani (1986) [24]"   74 13  4  8  49 .29166666 43 64 45 46 53 3.5   6 3.5   8   7
"Dura (1997) [25]"         48  3  4 14  27         1 86 89 85 77 79  12  12  12  12  12
"Evans (1999) [26]"       113  2 10 17  84  .5833333 50 68 48 42 49   7 7.5   5 4.5   4
"Foresman (2001) [27]"    139 24 43 25  47  .9166667 79 82 79 73 76  11  11  11  11  11
"Mage (1989) [28]"        122 22  5 19  76 .08333334 36 57 42 35 42   1 1.5 1.5   1   1
"Mahant (2002) [29]"      162 14 30 21  97  .8333333 64 75 73 65 68  10  10  10  10  10
"Morin (1999) [30]"        70 20 41  2   7  .5833333 50 68 55 42 52   7 7.5   7 4.5 5.5
"Muensterer (2002) [31]"  386 35 76 34 241       .75 57 71 58 46 55   9   9 8.5   8   8
"Oostenbrink (2000) [32]" 140 21 20 16  83 .29166666 43 61 42 42 47 3.5   4 1.5 4.5   3
"Salih (1994) [33]"        42 26  3  1  12 .29166666 43 57 52 46 58 3.5 1.5   6   8   9
"Tan (1988) [34]"          55  3  6 14  32  .5833333 50 61 58 42 52   7   4 8.5 4.5 5.5
"Verber (1988) [35]"       62  8  9 20  25 .29166666 43 61 45 38 46 3.5   4 3.5   2   2
end

using the following code:

metan a b c d,  qe(q3) study( study ) or counts forestplot(astext(85) boxscale(65) spacing(0.95) leftjustify)

The forestplot does not display and the following error displays:

option qwt() not allowed

Error in forestplot

Any thoughts?

Regards
Suhail

[David Fisher]

Hi Suhail,
Ah, I knew someone would find a bug sooner or later :-)
Thankyou, I'll look into it. I assume this doesn't happen with admetan?
Best wishes,
David.

ETA: It was a very simple fix! I'll see what other communication I receive, but I'll try and get a small update available very soon. Sorry again!

[Suhail Doi]

Hi David,

That's right, admetan works fine

Many thanks

Suhail

[Suhail Doi]

Hi David,

Thanks for sorting out the earlier issue so quickly!

Another more complicated request - feel free to ignore :-)

I note that proportions have been added to the metan suite and the Freeman-Tukey double arc sin transformation implemented. Also I note that you used the Schwarzer et al implementation of the Miller back transform that uses the harmonic mean [1]. They illustrate the issues with this implementation using the dataset below:

* Example generated by -dataex-. To install: ssc install dataex
clear
input str3 studyname long n int cases
"S1"  217154 422
"S10"  16557  32
"S13"    676   1
"S18"     44   1
"S26"     29   1
end

and report a 0 (0, 0) RE result which also is reported by metan and is counter-intuitive. MetaXL reports the RE result from this dataset as:

0.199%

(0.137%,

0.272%)

using the same transformation. The difference is because after extensive simulations we realized that for meta-analysis we should use 1/V in place of use of harmonic N for back conversion as that was not meant for meta-analysis [2]. However, we must use the simple back-conversion, (SIN(t/2))^2, when P/V or (1-P)/V) is less than 2 to avoid the back-transform from bombing out [V is FTukey pooled variance t is the FTukey pooled proportion and P is the simple back transformed proportion given by (SIN(t/2))^2) ]. Thus based on the values P/V or (1-P)/V the Miller back transform is either used (but N replaced by 1/V) or the simple back transform , P, is retained. For the CI's, we replace the lower by 0 if P/V<2 (otherwise go for the modified Miller backtransform) and the upper by 1 if (1-P)/V <2 (otherwise go for the modified Miller back-transform). All this is explained in our paper [2].

Would it be possible to add the MetaXL implementation as an option (I can confirm through extensive simulations that it outperforms all other options) Also we use the variance of the transformed proportion for individual studies as 1/(N+0.5) and not what has been reported by Schwarzer et al and that will be needed to pair with this option if you decide to add this as documented in our paper [2].

Regards
Suhail

References

1. Schwarzer G, Chemaitelly H, Abu-Raddad LJ, Rücker G. 2019. Seriously misleading results using inverse of Freeman-Tukey double arcsine transformation in meta-analysis of single proportions. Research Synthesis Methods 10: 476–483.

2. Barendregt JJ, Doi SA, Lee YY, Norman RE, Vos T. Meta-analysis of prevalence. J Epidemiol Community Health. 2013 Nov 1;67(11):974-8

[Nicoletta Riva]

Hi David,

Nice package the updated metan!
I especially like the inclusion of metaprop.

I have tried metan both with proportion and with rr, I get the tables and the stats correctly, but I do not manage to create the forest plots (I am using STATA 12).

Just to give you an example, this is my dataset:

* Example generated by -dataex-. To install: ssc install dataex
clear
input str14 study byte(deadGrp1 totalGrp1 deadGrp2 totalGrp2) float(aliveGrp2 aliveGrp1)
"study1" 1 15 5 20 15 14
"study2" 2 60 4 64 60 58
"study3" 3 22 3 24 21 19
"study4" 4 36 2 40 36 32
"study5" 5 45 1 44 43 40
end

these are the two commands that I am running:

metan deadGrp1 totalGrp1, proportion random ftt lcols(study) counts forestplot

or

metan deadGrp1 aliveGrp1 deadGrp2 aliveGrp2, rr random lcols(study) forestplot

in the first case, I get

Studies included: 5
Participants included: 178

Meta-analysis pooling of Freeman-Tukey transformed Proportions
using the random-effects inverse-variance model
with DerSimonian-Laird estimate of tau²

--------------------------------------------------------------------
study                | Proportion  [95% Conf. Interval]   % Weight
---------------------+----------------------------------------------
study1               |     0.067      0.012     0.298       8.80
study2               |     0.033      0.009     0.114      33.01
study3               |     0.136      0.047     0.333      12.70
study4               |     0.111      0.044     0.253      20.34
study5               |     0.111      0.048     0.235      25.15
---------------------+----------------------------------------------
Overall, DL          |     0.077      0.038     0.125     100.00
--------------------------------------------------------------------

Test of overall effect = 0:  z =   5.597  p = 0.000


Heterogeneity measures, calculated from the data
with Conf. Intervals based on Gamma (random-effects) distribution for Q
---------------------------------------------------------
Measure              |     Value      df      p-value
---------------------+-----------------------------------
Cochran's Q          |      4.13        4      0.389
                     |            -[95% Conf. Interval]-
H                    |     1.016     1.000     1.695
I² (%)               |      3.0%      0.0%     65.2%
---------------------------------------------------------
H = relative excess in Cochran's Q over its degrees-of-freedom
I² = proportion of total variation in effect estimate due to between-study heterogeneity (based on Q)


Heterogeneity variance estimates
-----------------------------------
Method               |     tau²
---------------------+-------------
DL                   |    0.0002
-----------------------------------
invalid syntax
Error in metan.BuildResultsSet
(Note: meta-analysis model was fitted successfully)
r(197);

it gives Error in metan.BuildResultsSet and does not produce the forestplot. Same error in the second case.

I was wondering what is my mistake. Do you have any suggestions?

Thanks in advance!

Nicoletta

[David Fisher]

Hi Nicoletta Riva ,

Thankyou for your message. I have received communication from another user recently, using Stata 13, who reported the same problem. I'm fairly sure the reason is a subtle change to the way Stata works, made at the time of Stata 14. This means that -metan- currently does not work properly with Stata 13 and below -- many apologies for this!! Thankfully the solution is simple, and I will be releasing an update very soon.

In the meantime, if you have some urgent work to complete, let me know and I will try to help (e.g. I can create the plot at my end if you are happy to share the data, and/or I can send you an "unofficial" updated version of the code.)

Best wishes,

David.

[Nicoletta Riva]

Hi David,

Thanks for your prompt reply.

Temporarily, I went back to the previous commands (admetan and metaprop, respectively) and the forest plots are created correctly. I'm looking forward to see the update.

Thanks again for your help!!

Nicoletta

[Luis Furuya-Kanamori]

Hi David,
Thanks for updating metan.
I tried to use the Freeman-Tukey double-arcsine transformation and there may be a glitch.

use http://fmwww.bc.edu/repec/bocode/m/metan_example_data, clear
metan cdeath csample, proportion transform(ftukey, iv) label(namevar=id)

gives the error message " 'ivariance' found where number expected "


Kind regards,
Luis

[David Fisher]

With thanks as ever to Kit Baum, an updated version of metan (v4.02)is now available via SSC.

The bugs identified by Nicoletta Riva and by Luis Furuya-Kanamori have now (hopefully!) been fixed.

Many thanks to daniel klein for suggesting a solution to the first bug and for help with testing under Stata 11.2 and 12.1, and to Suhail Doi for helping me to track down the second bug.

Additionally, the method described in post #5 is now implemented as: metan [varlist], proportion transform(ftukey, iv)

Best wishes to all,

David.

[Luis Furuya-Kanamori]

Many thanks David, it is working well now

Luis

[Suhail Doi]

Hi David,

Just noted that the cumulative command on the proportions forest plot in metan and on the table output only changes the weights but not the rest of the plot or the table output.

Regards
Suhail

Edit: It does work as intended if we omit the tr(xyz , x) command

[Carole Khairallah]

Thank you for the update.
However, since I updated my ado files, I get a warning when using ipdmetan with aggregated data from 1 study, while I could get estimates from that particular study before.

Note: 1 study with missing or insufficient data found;  use the keepall option to include this study in the summary table and forest plot

Also, adding the option "keepall" to show the study that was excluded from the model returns an error

>1 invalid name
Error in ipdmetan.metan_setup

Is there a way to run an earlier version of ipdmetan? It may not be related to the update, but I can't think of anything else...I am using Stata v16.1.
Please see below the syntax I use for ipdmetan

 ipdmetan, study(study) ad(Data\temp\nigeria_bin21.dta, byad vars(_ES _lci _uci)) saving(Data\temp\bin21.dta, replace): glm bin21 arm01 i.Grav3Gr, fam(bin) link(log) eform

Would you have any suggestion? Thanks in advance for your help.
Best regards,
Carole

[Nicoletta Riva]

Thanks a lot David!!

metan [varlist], proportion transform(ftukey, iv) works great!

Best regards,

Nicoletta

[David Fisher]

Nicoletta Riva Thanks for the feedback, great to hear!

Carole Khairallah :
Thanks for the bug report. I think I have identified and fixed it (privately), but of course I am only working with my own data and not with yours. Also, I am still unsure what is going on when you say "I could get estimates from that particular study before". Therefore, could you please do one of the following things for me?
(1) Send me (via Private Message) a copy of your working data and code which reproduces the error. Anonymised and/or sampled if necessary, just so long as the error is present.
or (2) Send me a Private Message with a contact email address for you, so that I can send you a corrected version of the program which you could run on your data and feed-back on whether it works or not.

Many thanks, and best wishes,

David.