Updated version of -ipdmetan- on SSC, plus an -ipdover- example comparing effects with reference categories

[David Fisher]

With thanks as ever to Kit Baum, an updated version of the ipdmetan package is now available for download from SSC (ssc install ipdmetan, or adoupdate ipdmetan if already installed). The package description from SSC is given below.

This version (1.07) includes the following bug-fixes and additions:

Bug fixes:
- Fixed bug preventing use of nohet option
- Fixed bug in implementation of "empirical Bayes" random-effects estimator
- forestplot no longer uses char(160) to represent blank (non-breaking) space due to differences in character mapping across systems
- Fixed bug in forestplot relating to varnames in lcols() and rcols() not having variable labels (the “must specify at least one of target or maxwidth” error)
- Now correctly parses “...if varname=="label":lblname...” syntax

Changes/additions:
- Added cumulative IPD meta-analysis functionality
- Added Kenward-Roger random-effects estimator, using an expression for the two-stage model derived from their paper (Biometrics 1997; 53: 983-97)
- Improvements to aspect ratio and text sizing in forestplot, including saving important "dimensions" in r() and optionally saving them to a matrix to be applied to subsequent plots

I am also happy to be able to draw your attention to the current issue of the Stata Journal, where you can find an article discussing ipdmetan and forestplot in more detail (Vol. 15, No. 2, pp. 369-96).

One addition which I have not yet had the time to implement is the creation of forestplots of “prognostic” subgroup effects -- that is, where the outcome in each level of a subgroup is compared to a reference (baseline) category. The current implementation only allows the effect of a treatment variable to be plotted within each subgroup level. Since I have had so many requests for this functionality, below is an example of a “hack” which can be used to achieve this effect in the meantime. I hope this will prove useful.

Many thanks,

David.

David Fisher
Statistician
MRC Clinical Trials Unit at UCL
e-mail: [email protected]


SSC DESCRIPTION

A set of routines for conducting two-stage individual participant meta-analysis, and associated functionality. The main routine, ipdmetan, loops over a series of categories, fits the desired model to the data within each, and generates pooled effects, heterogeneity statistics etc, as appropriate. Various random-effects subroutines are available, and aggregate data may be included from an external dataset. The routine forestplot is a stand-alone, re-written and extended version of the graphics routine within the existing SSC command metan. ipdover allows ipdmetan to be used outside of the meta-analytic context, for example for creating a forestplot of a series of (potentially overlapping) subgroups within a single study. admetan allows ipdmetan to be run on aggregate data only, if currently in memory, as a direct alternative to metan. Finally, petometan is an extension of the built-in command sts test for performing meta-analysis on binary or time-to-event data using the Peto (log-rank) method.


*********************

EXAMPLE CODE FOR COMPARING OUTCOMES WITHIN SUBGROUPS TO A REFERENCE CATEGORY

* Load data
use "http://fmwww.bc.edu/repec/bocode/i/ipdmetan_example", clear
stset tcens, fail(fail)
 
* Generate indicator vars for each level of "stage"
qui tab stage, gen(stage)
 
* Run -ipdover- repeatedly to save the different parts we need
* (N.B. the first run uses "stcox trt" simply to create the desired structure in the saved dataset; the actual coefficients are not ultimately used)
ipdover, over(stage) over(trialid) hr nosubgroup nooverall nograph saving(stage_all, replace) : stcox trt
ipdover, over(trialid) hr nooverall nograph poolvar(2.stage) saving(stage2, replace) rcols((sum) stage1 stage2) : stcox i.stage
ipdover, over(trialid) hr nooverall nograph poolvar(3.stage) saving(stage3, replace) rcols((sum) stage3) : stcox i.stage
 
* Now load the saved datasets and assemble
use stage_all, clear
drop if _USE==1
append using stage2 stage2 stage3, gen(dataid)
 
* Sort out arrangement of trials and covariate levels
replace _BY = _LEVEL if dataid > 0
replace _LEVEL = dataid if dataid > 0
replace _LABELS = "Stage I" if _LEVEL == 1
replace _LABELS = "Stage II" if _LEVEL == 2
replace _LABELS = "Stage III" if _LEVEL == 3
 
* Sort out numbers of patients
replace _NN = stage1 if dataid == 1
replace _NN = stage2 if dataid == 2
replace _NN = stage3 if dataid == 3
drop stage1 stage2 stage3
 
* Replace effect size with null in reference category
* (N.B. I couldn't do this directly because -ipdover- would complain that there was "no data")
replace _ES = 0 if dataid == 1
replace _seES = 0 if dataid == 1
replace _LCI = 0 if dataid == 1
replace _UCI = 0 if dataid == 1
 
* Manually generate "effect size" string, including for reference category
gen str estText = string(exp(_ES), "%10.2f") + " (" + string(exp(_LCI), "%10.2f") + ", " + string(exp(_UCI), "%10.2f") + ")" if _USE == 1
replace estText = "1.00 (Reference)" if _LEVEL == 1
label var estText "Haz. ratio (95% CI)"
 
* Sort data and produce forestplot
sort _BY _USE _LEVEL
forestplot, hr nostat nowt rcols(estText _NN)

[David Fisher]

Updated version of -ipdmetan- on SSC; now has nearly all the functionality of -metan-


Happy Friday, Statalisters!

With thanks as ever to Kit Baum, an update of the ipdmetan package is now available for download from SSC (ssc install ipdmetan, or adoupdate ipdmetan if already installed). The package description from SSC is given below.


This is such a major update that I've up-versioned the package to v2.0. There are various improvements and a few minor bug fixes; but the bulk of what's new relates to incorporating the "four-element" and "six-element" syntaxes of metan into both admetan and ipdmetan/ipdover. In other words, admetan can now handle cell-count data (#a #b #c #d) and mean difference data (n, mean, SD by treatment arm) and can use methods such as Mantel-Haenszel and Standardised Mean Differences. Furthermore, ipdmetan/ipdover can also now handle data of this type, using an alternative syntax which uses collapse to form the appropriate aggregated data. Overall, the vast majority of metan functionality is now replicated within admetan, making this effectively the first update to the ever-popular metan package since September 2010. (In particular, admetan is far more likely to run successfully in resource-limited environments than metan; I've seen a couple of examples where users have hit the upper limit of permitted variables due to metan being inefficient with tempvars.)

Sadly, the time spent doing this has meant that I've still not gotten around to implementing some previously-requested functionalities, such as forest plots of prognostic subgroup effects and better continuity-correction options. However, now that this structural update is complete, I'll aim to get some of these done fairly soon.


Other notable additions/improvements:

- cumulative and "influence" meta-analysis, where the latter removes one study at a time from the overall calculation to assess that study's influence
- predictive distributions; this has long been available in metan but is now also available in ipdmetan/admetan
- more control over formatting in forest plots, e.g. x-axis leading zeros, string justification, buffering between data columns, line breaks in column headings.

The main omissions in admetan functionality relative to metan are the first(), firststats(), second() and secondstats() options which allowed, for instance, both fixed- and random-effects estimates to be shown in the same forest plot. I will consider adding these in a future update, but note that the way forestplot is set up means that, with a bit of additional work, these features and many others may be recreated by saving, loading and manipulating the forest plot dataset (or "results set"). The admetan help file, which recreates the examples given in the metan help file, demonstrates one way to recreate the functionality of first(), firststats(), second() and secondstats().


Lastly, if I may, I will take the opportunity to promote my recent-published article in BMJ Research Methods & Reporting on the subject of treatment-covariate interactions and subgroups, and why a "within-trials only" or "deft" approach is preferable. One way of implementing this approach is to use ipdmetan with the interaction option.


Many thanks,

David.



============================================

SSC DESCRIPTION

IPDMETAN: A set of routines for conducting two-stage individual participant meta-analysis, aggregate (summary) data meta-analysis, and for creating high-quality forest plots. The two-stage routine, ipdmetan, loops over a series of categories, fits the desired model to the data within each, and generates pooled effects, heterogeneity statistics etc, as appropriate; aggregate data may also be included from an external dataset. The aggregate-data routine, admetan, carries over much of the functionality of the existing SSC command metan, but with a wider range of random-effects and variance-correction models. There is also an “immediate” version, admetani. The routine forestplot is a stand-alone, re-written and extended version of the graphics routine within the existing SSC command metan. ipdover allows ipdmetan to be used outside of the meta-analytic context, for example for creating a forest plot of a series of (potentially overlapping) subgroups within a single study.

[Alexander Rodriguez]

Hi David,
I am a recent user (about 1 year experience) of Stata v14.2 though this is my first post.

I have been playing around with your package ipdover which I find very useful. However, recently my code keeps returning the r(9) assumptions are false error.

I am using ipdover to plot HR(95%CI) for mortality after Cox regression. I am stratifying the cohort according to four different levels of calcium and vitamin D [ipdover, over(calc) over(vitd)...]. The calcium groups are defined according to arbitary cut-points. However when I change the calcium variable to another one that uses clinically relevant calcium cut-points [ipdover, over(calc_2) over(vitd)...] the r(9) occurs.

Do you have a solution for this because if that assumptions are satisfied using one grouping of calcium there should be no reason for the assumptions to be violated with the other grouping. I am happy to provide you with code privately if my explanation here isn't clear.

Many thanks,
Alexander

[David Fisher]

Hi Alexander,

Thanks for your message; sorry you're having trouble.

Does the error message say "Error in ipdmetan" or similar? If so, could you tell me the complete error message? (that is, everything in red).

Also, are you familiar with Stata's debugging commands "set trace on" and "set tracedepth #"? If so, could you copy-and-paste a few (say 10) of the lines immediately preceding the error, so I can see exactly where it occurred?

Is there anything "extreme" about the calc_2 variable? e.g. does it contain a very small subgroup? Does it vary within the observations set up for survival analysis (i.e. stset)?

Thanks,

David.

[David Fisher]

Updated version of -ipdmetan- on SSC

With thanks as ever to Kit Baum, version 2.1 of the ipdmetan package is now available for download from SSC (ssc install ipdmetan, or adoupdate ipdmetan if already installed).

A reminder that admetan (part of this package) now does pretty much everything metan does, plus a whole lot more!

This is mostly a bug-fixing update, ironing out some problems which slipped through the net in v2.0 (including the issue raised by Alexander Rodriguez in previous posts in this thread).

Just to note that if you are, or have been, making use of: (a) the "sensitivity" random-effects model with user-specified I-squared; or (b) the returned statistic r(Isq), that admetan now takes in and returns I-squared values on the percentage scale; that is, values between 0 and 100. This mirrors the behaviour of metan and metaan. Apologies for any confusion this may cause.

Finally: Just to mention admetani, which is a new quasi-immediate version of admetan which works on directly-inputted values or existing matrices. The intended use cases are: (a) performing quick-and-dirty meta-analyses of a small number of studies whose results are to hand but which are not stored in the dataset in memory; and (b) performing meta-analyses based on beta-coefficients & covariance matrices from previously-run regression commands or from margins (as such, there may be some overlap with commands such as marginsplot and coefplot, but I've not explored these possibilities in detail yet).

Many thanks,

David.

[Marcos Almeida]

Thank you for the updating and congratulations for the achievement!

The fact of "mirroring" metan whenever feasible and at the same time giving much more in terms of "facilities" and craftsmanship will surely entice those ones basically acquainted wth metan to embrace ipmetan, like me.

Thanks again.

[David Fisher]

Thankyou very much, Prof. Almeida!

[Alexander Rodriguez]

Updated version of -ipdmetan- on SSC

With thanks as ever to Kit Baum, version 2.1 of the ipdmetan package is now available for download from SSC (ssc install ipdmetan, or adoupdate ipdmetan if already installed).

A reminder that admetan (part of this package) now does pretty much everything metan does, plus a whole lot more!

This is mostly a bug-fixing update, ironing out some problems which slipped through the net in v2.0 (including the issue raised by Alexander Rodriguez in previous posts in this thread).

Just to note that if you are, or have been, making use of: (a) the "sensitivity" random-effects model with user-specified I-squared; or (b) the returned statistic r(Isq), that admetan now takes in and returns I-squared values on the percentage scale; that is, values between 0 and 100. This mirrors the behaviour of metan and metaan. Apologies for any confusion this may cause.

Finally: Just to mention admetani, which is a new quasi-immediate version of admetan which works on directly-inputted values or existing matrices. The intended use cases are: (a) performing quick-and-dirty meta-analyses of a small number of studies whose results are to hand but which are not stored in the dataset in memory; and (b) performing meta-analyses based on beta-coefficients & covariance matrices from previously-run regression commands or from margins (as such, there may be some overlap with commands such as marginsplot and coefplot, but I've not explored these possibilities in detail yet).

Many thanks,

David.

Thanks David!

[Shixiang Wang]

Lastly, if I may, I will take the opportunity to promote my recent-published article in BMJ Research Methods & Reporting on the subject of treatment-covariate interactions and subgroups, and why a "within-trials only" or "deft" approach is preferable. One way of implementing this approach is to use ipdmetan with the interaction option.

Dear David,

Can you provide an example how to implement "def" approach using ipdmetan and generate plots like Fig 2 in this published article?

Best,

Shixiang

[David Fisher]

For the benefit of Forum readers: Shixiang has emailed me directly with a sample of his data, and I have given some help.
However, I agree that the BMJ paper does not make it clear how the plots were generated, especially in the context of aggregate data. I shall aim to include an example of this sort in the documentation of the next update of admetan/ipdmetan.
Thanks,
David.

[David Fisher]

Hi again,

The next of update of admetan / ipdmetan is now available; see https://www.statalist.org/forums/for...alysis-command. I haven’t included a BMJ-style example (as requested by Shixiang Wang above) in the command documentation, as I’d already spent too long on the code and wanted to make it available. So instead, here is a “bonus” example.

Data are taken from Analysis 8.2 of the Cochrane review “Services for reducing duration of hospital care for acute stroke patients” (Fearon et al 2012), available at: https://www.cochranelibrary.com/cdsr...0443.pub3/full (as this is available Open Access, I’m assuming it’s ok to recreate it here, but do let me know if otherwise!)

* Example generated by -dataex-. To install: ssc install dataex
clear
input byte(trial carer) int n1 float(mean1 sd1) int n0 float(mean0 sd0)
 1 1  24 19.5 20.21  25 37.4 25.35
 4 1  38 44.4 25.48  38 42.8 45.05
 7 1 116 30.7 32.26 101 43.1    44
 8 1   6 25.6 33.64   7   56 43.12
 9 1  58  9.8  5.32  56   12  7.07
10 1  24 18.2 22.05  22 33.8 36.05
11 1  18 19.7 13.75  15 23.4  14.6
12 1  30 12.6  6.59  27 33.5 30.97
14 1  20 22.2  34.1  11 28.3  50.4
 1 2  18 22.7 21.44  19 34.1 22.73
 4 2  21 37.5 32.89  16 65.4 48.97
 7 2  49 37.9 34.68  62 38.4 32.88
 8 2   4 48.5 42.79   4   15  7.87
10 2  20 25.8 27.33  20 33.9 35.21
11 2  24 31.5 18.26  25 40.1 23.26
12 2  12 15.9  7.48  14   21   9.8
14 2  11 25.9  23.8  15 32.9  39.5
end
label values trial TRIAL
label def TRIAL 1 "Adelaide 2000", modify
label def TRIAL 4 "Belfast 2004", modify
label def TRIAL 7 "London 1999", modify
label def TRIAL 8 "Manchester 2001", modify
label def TRIAL 9 "Montreal 2000", modify
label def TRIAL 10 "Newcastle 1997", modify
label def TRIAL 11 "Oslo 2000", modify
label def TRIAL 12 "Stockholm 1998", modify
label def TRIAL 14 "Trondheim 2004", modify
label values carer CARER
label def CARER 1 "Carer present", modify
label def CARER 2 "No carer", modify
label var trial "Trial name"
label var carer "subgroup"

Our first task is to create a dummy observation representing the missing “No carer” category for the Montreal trial:

expand 2 if trial=="Montreal 2000":TRIAL
replace carer = 2 in L
foreach v of varlist n1-sd0 {
     replace `v' = . in L
}
sort carer trial

Now we can proceed to create the left-hand plot of Figure 2 of the BMJ paper, saving the “dimensions” in a matrix to be applied later to the right-hand plot:

graph drop _all
admetan n1 mean1 sd1 n0 mean0 sd0, wmd ovwt by(trial) study(carer) nooverall nosubgroup keepall keeporder ///
forestplot(nostats nowt favours("Favours" "ESD" " " # "Favours" "no ESD" " ") xlabel(-50 0 50, force) astext(35) savedims(A) name(fig2a))

Creating the right-hand plot is a little trickier. We begin by analysing the interactions. For each trial, the interaction is formed from two effect estimates, one from each subgroup. We could use generate and replace to form the difference in effect sizes and the sum of their variances; or we could take a short-cut using ipdmetan and vwls (see help vwls to see what this is doing):

ipdmetan, study(trial) keepall keeporder effect("Diff. in mean diffs.") saving(fig2b, replace) nogr ///
     : vwls _ES ib1.carer, sd(_seES)

Next, we load the “forestplot results set” and use expand to pad out the data to match the structure of the left-hand plot. (This can be tricky to envisage, and may involve some trial-and-error.) Finally, we create the right-hand plot, applying the same “dimensions” as in the left-hand plot:

use fig2b, clear
gen toexpand = 1 + 3*inlist(_USE, 1, 2) + 2*(_USE==5)      // 2 new obs per trial, plus 1 for overall
local oldN = _N
expand toexpand, gen(expand)
replace _USE = 6 * (1 - (_n > `oldN' & !mod(_n, 3))) if expand
replace _WT = . if _USE==0
sort _STUDY _USE
forestplot, useopts interaction usedims(A) nonames ///
     favours("Favours greater effect" "of ESD with" "carer present" # "Favours greater effect" "of ESD with" "no carer present", fp(40)) ///
     xlabel(-50 0 50, force) name(fig2b)

Finally, we use graph combine:

graph combine fig2a fig2b, imargin(zero) xsize(4)

I hope this is useful to the community.

Thanks,

David.

[Elen Xr]

Hello,

I am running a meta-analyses by metan command and I have high levels of heterogeity (I sq=72% p=.003). Until, I read for the admetan command, so i decide to run it again with this command and then i got I sq=19%. I would like to ask, is there any basic different in the way of comparisons (I have read the difference but if there is something that i dont understand)?? Beacause the difference is high between two commands...It's my first meta-analyses with heterogeneity and i dont know what to do exactly or what results should i trust

Thank you!

[David Fisher]

Hi Elen,
I agree that such a large difference is unusual.
Could you possibly show me your results exactly as the appear on your screen, along with the exact commands you used? I know sometimes sharing the data can be a problem, but hopefully in this case the commands and the output alone will give us a clue.
The simplest way of doing it is to highlight the text, and copy-and-paste it into a reply box here. If you then surround it with markers:

 your text here

it will be easier to read.
Many thanks,
David.

[Elen Xr]

Hi, David,

First of all, before i told you or sent you some copy-paste from my data...is there a differnce between the metan command or metaan ?? Because I run it with both of them and again i have dieffernt results with a large diefference,

Thank you!

[David Fisher]

Hi Elen,

There should not be any difference between commands *if* the same models/tests are being requested. However, similar commands such as metan, admetan and metaan sometimes have different syntaxes and defaults, so you may have to write out your command in different ways to obtain the same results. Does that make sense?

This is why I asked if you could show me your commands and your on-screen results to begin with, rather than your actual data.

Many thanks,

David.