"No trimming performed" with the evidence of publication bias?
Hello,
I have a question about the use of 'metatrim' command and interpretation of the results. I would be grateful if you could advise me. Below I have indicated the steps of my analysis and the output. I used Stata version 14.
Step 1- I performed random effects meta analysis on double arcsine transformed proportions.
Step 2- Then I performed metabias command to investigate the publication bias
Step 3- Results of the both tests (Begg's, Egger's) indicate the evidence of publication bias. Hence, I performed metatrim command using below mentioned codes.
metatrim _ES _seES, reffect print graph funnel id(Authorsandyearofpublication)
metatrim _ES _seES, reffect mixed print graph funnel id(Authorsandyearofpublication)
Results were identical at both stages. The Stata output says no trimming performed; data unchanged. However, the random effect pooled estimate (red font) is different to the original pooled estimate I received from metaprop command. What would be the possible reasons? In which occasions trimming is not performed?
Step 4- Then I performed metatrim command with the 'flip' option and both 'flip' and 'mixed' options. Then trimming has performed but results are different. I wonder which option to be used in my case (proportion meta-analysis)
Please see below for all the steps with results.
Step 1- I performed random effects meta analysis on double arcsine transformed proportions. Below is the Stata code I used and the output of the analysis.
metaprop Frailty Effectivesample , lcols( Authorsandyearofpublication Effectivesample Frailty ) random ftt xlabel (0,0.2,0.4,0.6,0.8,1) xtitle("ES=Prevalence of frailty",size(2))
Study | ES [95% Conf. Interval] % Weight
---------------------+---------------------------------------------------
Tribess et al, 2012 | 0.20 0.17 0.23 1.48
Junior et al, 2014 | 0.24 0.19 0.29 1.46
Pegorari et al, 2014 | 0.13 0.11 0.15 1.49
Santos et al, 2015 | 0.17 0.12 0.24 1.41
Closs et al, 2016 | 0.21 0.18 0.25 1.47
Mello et al, 2017 | 0.12 0.08 0.19 1.41
de Albuquerque Sousa | 0.17 0.14 0.21 1.47
dos Santos Amaral et | 0.19 0.15 0.23 1.46
Moreira et al, 2013 | 0.10 0.08 0.12 1.48
Neri et al, 2013 (Be | 0.11 0.09 0.13 1.48
Neri et al, 2013 (Pa | 0.10 0.07 0.13 1.47
Neri et al, 2013 (Ca | 0.09 0.06 0.12 1.47
Neri et al, 2013 (Po | 0.09 0.07 0.13 1.47
Neri et al, 2013 (Er | 0.08 0.06 0.11 1.47
Neri et al, 2013 (Ca | 0.08 0.06 0.10 1.48
Neri et al, 2013 (Iv | 0.09 0.05 0.13 1.44
Vieira et al, 2013 | 0.09 0.07 0.11 1.48
Ricci et al, 2014 | 0.10 0.08 0.12 1.48
Calado et al, 2016 | 0.09 0.07 0.12 1.47
Silveira et al, 2015 | 0.11 0.05 0.22 1.30
Augusti et al, 2017 | 0.22 0.17 0.27 1.46
Ferriolli et al, 201 | 0.12 0.10 0.15 1.48
Ferriolli et al, 201 | 0.16 0.12 0.19 1.47
Ferriolli et al, 201 | 0.10 0.08 0.13 1.47
Ocampo-Caparro et al | 0.13 0.09 0.17 1.46
Curcio et al, 2014 | 0.12 0.11 0.14 1.49
Samper-Ternent et al | 0.09 0.08 0.11 1.49
Sanchez-Garcia et al | 0.11 0.10 0.13 1.49
Moreno-Tamayo et al, | 0.12 0.10 0.15 1.48
Chen et al, 2015 | 0.13 0.10 0.16 1.48
Wu et al, 2017 | 0.06 0.06 0.07 1.50
Dong et al, 2017 | 0.04 0.03 0.05 1.49
Wang et al, 2015 | 0.14 0.11 0.19 1.46
Badrasawi et al, 201 | 0.09 0.07 0.12 1.47
Kashikar et al, 2016 | 0.10 0.07 0.15 1.45
Gurina et al, 2011 | 0.21 0.18 0.25 1.48
Alvarado et al, 2008 | 0.27 0.24 0.29 1.49
Alvarado et al, 2008 | 0.41 0.38 0.43 1.49
Alvarado et al, 2008 | 0.43 0.40 0.45 1.49
Alvarado et al, 2008 | 0.39 0.37 0.41 1.49
Alvarado et al, 2008 | 0.40 0.37 0.42 1.49
Aguilar-Navarro et a | 0.37 0.36 0.39 1.50
Avila-Funes et al, 2 | 0.14 0.12 0.17 1.48
Sanchez-Garcia et al | 0.16 0.14 0.17 1.49
Jotheeswaran et al, | 0.08 0.06 0.10 1.49
Jotheeswaran et al, | 0.09 0.07 0.11 1.49
Jotheeswaran et al, | 0.21 0.19 0.23 1.49
Jotheeswaran et al, | 0.35 0.32 0.37 1.49
Jotheeswaran et al, | 0.11 0.09 0.14 1.48
Jotheeswaran et al, | 0.10 0.08 0.12 1.48
Jotheeswaran et al, | 0.08 0.07 0.10 1.48
Jotheeswaran et al, | 0.26 0.24 0.28 1.49
Jotheeswaran et al, | 0.17 0.14 0.21 1.47
Jotheeswaran et al, | 0.11 0.10 0.13 1.49
Zhu et al, 2016 | 0.12 0.10 0.14 1.49
Akin et al, 2015 | 0.28 0.25 0.31 1.48
Fhon et al, 2012 | 0.39 0.33 0.45 1.45
Agreli et al, 2013 | 0.30 0.22 0.40 1.39
Duarte et al, 2013 | 0.39 0.32 0.47 1.43
Del Brutto et al, 20 | 0.31 0.26 0.37 1.46
Fabrico-Wehbe et al, | 0.31 0.24 0.40 1.41
Carneiro et al, 2016 | 0.41 0.37 0.46 1.47
Sathasivam et al, 20 | 0.06 0.04 0.08 1.48
Perez-Zepeda et al, | 0.45 0.44 0.46 1.50
Woo et al, 2015 (Urb | 0.17 0.16 0.18 1.50
Woo et al, 2015 (Rur | 0.05 0.04 0.07 1.49
Galban et al, 2009 | 0.51 0.47 0.56 1.48
Boulos et al, 2016 | 0.36 0.34 0.39 1.49
---------------------+---------------------------------------------------
Random pooled ES | 0.17 0.14 0.21 100.00
---------------------+---------------------------------------------------
Heterogeneity chi^2 = 8536.65 (d.f. = 67) p = 0.00
I^2 (variation in ES attributable to heterogeneity) = 99.22%
Estimate of between-study variance Tau^2 = 0.12
Test of ES=0 : z= 19.36 p = 0.00
Step 2- Then I performed metabias command to investigate the publication bias
metabias _ES _seES, graph(egger)
Note: default data input format (theta, se_theta) assumed.
Tests for Publication Bias
Begg's Test
adj. Kendall's Score (P-Q) = 399
Std. Dev. of Score = 188.91 (corrected for ties)
Number of Studies = 68
z = 2.11
Pr > |z| = 0.035
z = 2.11 (continuity corrected)
Pr > |z| = 0.035 (continuity corrected)
Egger's test
------------------------------------------------------------------------------
Std_Eff | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
slope | .2911986 .0352365 8.26 0.000 .2208467 .3615505
bias | -2.72589 1.164169 -2.34 0.022 -5.050229 -.4015514
------------------------------------------------------------------------------
Step 3- Results of the both tests indicate the evidence of publication bias. Hence, I performed metatrim command.
metatrim _ES _seES, reffect print graph funnel id(Authorsandyearofpublication)
metatrim _ES _seES, reffect mixed print graph funnel id(Authorsandyearofpublication)
I used above mentioned codes without and with the code 'mixed'. Results were identical at both stages. The Stata output says no trimming performed; data unchanged. However, the random effect pooled estimate (red font) is different to the original pooled estimate received from metaprop command. What would be the possible reasons? In which occasions trimming is not performed?
Note: default data input format (theta, se_theta) assumed.
Meta-analysis
| Pooled 95% CI Asymptotic No. of
Method | Est Lower Upper z_value p_value studies
-------+----------------------------------------------------
Fixed | 0.218 0.211 0.225 59.430 0.000 68
Random | 0.187 0.153 0.222 10.657 0.000
Test for heterogeneity: Q= 1421.736 on 67 degrees of freedom (p= 0.000)
Moment-based estimate of between studies variance = 0.019
Trimming estimator: Linear
Meta-analysis type: Random-effects model initially, but mixed mode
iteration | estimate Tn # to trim diff
----------+--------------------------------------
1 | 0.218 832 0 2346
2 | 0.218 832 0 0
Note: no trimming performed; data unchanged
Filled
Meta-analysis
| Pooled 95% CI Asymptotic No. of
Method | Est Lower Upper z_value p_value studies
-------+----------------------------------------------------
Fixed | 0.218 0.211 0.225 59.430 0.000 68
Random | 0.187 0.153 0.222 10.657 0.000
Test for heterogeneity: Q= 1421.736 on 67 degrees of freedom (p= 0.000)
Moment-based estimate of between studies variance = 0.019
Step 4- Then I performed metatrim command with the 'flip' option and both 'flip' and 'mixed' options. Then trimming has performed but results are different. I wonder which option to be used in my case (proportion meta-analysis)
metatrim _ES _seES, reffect flip print graph funnel id(Authorsandyearofpublication)
Note: default data input format (theta, se_theta) assumed.
Meta-analysis
| Pooled 95% CI Asymptotic No. of
Method | Est Lower Upper z_value p_value studies
-------+----------------------------------------------------
Fixed | 0.218 0.211 0.225 59.430 0.000 68
Random | 0.187 0.153 0.222 10.657 0.000
Test for heterogeneity: Q= 1421.736 on 67 degrees of freedom (p= 0.000)
Moment-based estimate of between studies variance = 0.019
Trimming estimator: Linear
Meta-analysis type: Random-effects model
iteration | estimate Tn # to trim diff
----------+--------------------------------------
1 | 0.187 1300 4 2346
2 | 0.196 1364 6 128
3 | 0.200 1394 7 60
4 | 0.202 1405 7 22
5 | 0.202 1405 7 0
Filled
Meta-analysis
| Pooled 95% CI Asymptotic No. of
Method | Est Lower Upper z_value p_value studies
-------+----------------------------------------------------
Fixed | 0.234 0.227 0.240 68.002 0.000 75
Random | 0.202 0.170 0.234 12.338 0.000
Test for heterogeneity: Q= 1565.953 on 74 degrees of freedom (p= 0.000)
Moment-based estimate of between studies variance = 0.018
. metatrim _ES _seES, reffect flip mixed print graph funnel id(Authorsandyearofpublication)
Note: default data input format (theta, se_theta) assumed.
Meta-analysis
| Pooled 95% CI Asymptotic No. of
Method | Est Lower Upper z_value p_value studies
-------+----------------------------------------------------
Fixed | 0.218 0.211 0.225 59.430 0.000 68
Random | 0.187 0.153 0.222 10.657 0.000
Test for heterogeneity: Q= 1421.736 on 67 degrees of freedom (p= 0.000)
Moment-based estimate of between studies variance = 0.019
Trimming estimator: Linear
Meta-analysis type: Random-effects model initially, but mixed mode
iteration | estimate Tn # to trim diff
----------+--------------------------------------
1 | 0.218 1514 10 2346
2 | 0.248 1768 18 508
3 | 0.264 1912 22 288
4 | 0.270 1956 23 88
5 | 0.272 1966 23 20
6 | 0.272 1966 23 0
Filled
Meta-analysis
| Pooled 95% CI Asymptotic No. of
Method | Est Lower Upper z_value p_value studies
-------+----------------------------------------------------
Fixed | 0.272 0.266 0.279 83.859 0.000 91
Random | 0.257 0.223 0.291 14.820 0.000
Test for heterogeneity: Q= 2427.665 on 90 degrees of freedom (p= 0.000)
Moment-based estimate of between studies variance = 0.025
I look forward to hearing from you.
Thank you.
Deepani
I have a question about the use of 'metatrim' command and interpretation of the results. I would be grateful if you could advise me. Below I have indicated the steps of my analysis and the output. I used Stata version 14.
Step 1- I performed random effects meta analysis on double arcsine transformed proportions.
Step 2- Then I performed metabias command to investigate the publication bias
Step 3- Results of the both tests (Begg's, Egger's) indicate the evidence of publication bias. Hence, I performed metatrim command using below mentioned codes.
metatrim _ES _seES, reffect print graph funnel id(Authorsandyearofpublication)
metatrim _ES _seES, reffect mixed print graph funnel id(Authorsandyearofpublication)
Results were identical at both stages. The Stata output says no trimming performed; data unchanged. However, the random effect pooled estimate (red font) is different to the original pooled estimate I received from metaprop command. What would be the possible reasons? In which occasions trimming is not performed?
Step 4- Then I performed metatrim command with the 'flip' option and both 'flip' and 'mixed' options. Then trimming has performed but results are different. I wonder which option to be used in my case (proportion meta-analysis)
Please see below for all the steps with results.
Step 1- I performed random effects meta analysis on double arcsine transformed proportions. Below is the Stata code I used and the output of the analysis.
metaprop Frailty Effectivesample , lcols( Authorsandyearofpublication Effectivesample Frailty ) random ftt xlabel (0,0.2,0.4,0.6,0.8,1) xtitle("ES=Prevalence of frailty",size(2))
Study | ES [95% Conf. Interval] % Weight
---------------------+---------------------------------------------------
Tribess et al, 2012 | 0.20 0.17 0.23 1.48
Junior et al, 2014 | 0.24 0.19 0.29 1.46
Pegorari et al, 2014 | 0.13 0.11 0.15 1.49
Santos et al, 2015 | 0.17 0.12 0.24 1.41
Closs et al, 2016 | 0.21 0.18 0.25 1.47
Mello et al, 2017 | 0.12 0.08 0.19 1.41
de Albuquerque Sousa | 0.17 0.14 0.21 1.47
dos Santos Amaral et | 0.19 0.15 0.23 1.46
Moreira et al, 2013 | 0.10 0.08 0.12 1.48
Neri et al, 2013 (Be | 0.11 0.09 0.13 1.48
Neri et al, 2013 (Pa | 0.10 0.07 0.13 1.47
Neri et al, 2013 (Ca | 0.09 0.06 0.12 1.47
Neri et al, 2013 (Po | 0.09 0.07 0.13 1.47
Neri et al, 2013 (Er | 0.08 0.06 0.11 1.47
Neri et al, 2013 (Ca | 0.08 0.06 0.10 1.48
Neri et al, 2013 (Iv | 0.09 0.05 0.13 1.44
Vieira et al, 2013 | 0.09 0.07 0.11 1.48
Ricci et al, 2014 | 0.10 0.08 0.12 1.48
Calado et al, 2016 | 0.09 0.07 0.12 1.47
Silveira et al, 2015 | 0.11 0.05 0.22 1.30
Augusti et al, 2017 | 0.22 0.17 0.27 1.46
Ferriolli et al, 201 | 0.12 0.10 0.15 1.48
Ferriolli et al, 201 | 0.16 0.12 0.19 1.47
Ferriolli et al, 201 | 0.10 0.08 0.13 1.47
Ocampo-Caparro et al | 0.13 0.09 0.17 1.46
Curcio et al, 2014 | 0.12 0.11 0.14 1.49
Samper-Ternent et al | 0.09 0.08 0.11 1.49
Sanchez-Garcia et al | 0.11 0.10 0.13 1.49
Moreno-Tamayo et al, | 0.12 0.10 0.15 1.48
Chen et al, 2015 | 0.13 0.10 0.16 1.48
Wu et al, 2017 | 0.06 0.06 0.07 1.50
Dong et al, 2017 | 0.04 0.03 0.05 1.49
Wang et al, 2015 | 0.14 0.11 0.19 1.46
Badrasawi et al, 201 | 0.09 0.07 0.12 1.47
Kashikar et al, 2016 | 0.10 0.07 0.15 1.45
Gurina et al, 2011 | 0.21 0.18 0.25 1.48
Alvarado et al, 2008 | 0.27 0.24 0.29 1.49
Alvarado et al, 2008 | 0.41 0.38 0.43 1.49
Alvarado et al, 2008 | 0.43 0.40 0.45 1.49
Alvarado et al, 2008 | 0.39 0.37 0.41 1.49
Alvarado et al, 2008 | 0.40 0.37 0.42 1.49
Aguilar-Navarro et a | 0.37 0.36 0.39 1.50
Avila-Funes et al, 2 | 0.14 0.12 0.17 1.48
Sanchez-Garcia et al | 0.16 0.14 0.17 1.49
Jotheeswaran et al, | 0.08 0.06 0.10 1.49
Jotheeswaran et al, | 0.09 0.07 0.11 1.49
Jotheeswaran et al, | 0.21 0.19 0.23 1.49
Jotheeswaran et al, | 0.35 0.32 0.37 1.49
Jotheeswaran et al, | 0.11 0.09 0.14 1.48
Jotheeswaran et al, | 0.10 0.08 0.12 1.48
Jotheeswaran et al, | 0.08 0.07 0.10 1.48
Jotheeswaran et al, | 0.26 0.24 0.28 1.49
Jotheeswaran et al, | 0.17 0.14 0.21 1.47
Jotheeswaran et al, | 0.11 0.10 0.13 1.49
Zhu et al, 2016 | 0.12 0.10 0.14 1.49
Akin et al, 2015 | 0.28 0.25 0.31 1.48
Fhon et al, 2012 | 0.39 0.33 0.45 1.45
Agreli et al, 2013 | 0.30 0.22 0.40 1.39
Duarte et al, 2013 | 0.39 0.32 0.47 1.43
Del Brutto et al, 20 | 0.31 0.26 0.37 1.46
Fabrico-Wehbe et al, | 0.31 0.24 0.40 1.41
Carneiro et al, 2016 | 0.41 0.37 0.46 1.47
Sathasivam et al, 20 | 0.06 0.04 0.08 1.48
Perez-Zepeda et al, | 0.45 0.44 0.46 1.50
Woo et al, 2015 (Urb | 0.17 0.16 0.18 1.50
Woo et al, 2015 (Rur | 0.05 0.04 0.07 1.49
Galban et al, 2009 | 0.51 0.47 0.56 1.48
Boulos et al, 2016 | 0.36 0.34 0.39 1.49
---------------------+---------------------------------------------------
Random pooled ES | 0.17 0.14 0.21 100.00
---------------------+---------------------------------------------------
Heterogeneity chi^2 = 8536.65 (d.f. = 67) p = 0.00
I^2 (variation in ES attributable to heterogeneity) = 99.22%
Estimate of between-study variance Tau^2 = 0.12
Test of ES=0 : z= 19.36 p = 0.00
Step 2- Then I performed metabias command to investigate the publication bias
metabias _ES _seES, graph(egger)
Note: default data input format (theta, se_theta) assumed.
Tests for Publication Bias
Begg's Test
adj. Kendall's Score (P-Q) = 399
Std. Dev. of Score = 188.91 (corrected for ties)
Number of Studies = 68
z = 2.11
Pr > |z| = 0.035
z = 2.11 (continuity corrected)
Pr > |z| = 0.035 (continuity corrected)
Egger's test
------------------------------------------------------------------------------
Std_Eff | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
slope | .2911986 .0352365 8.26 0.000 .2208467 .3615505
bias | -2.72589 1.164169 -2.34 0.022 -5.050229 -.4015514
------------------------------------------------------------------------------
Step 3- Results of the both tests indicate the evidence of publication bias. Hence, I performed metatrim command.
metatrim _ES _seES, reffect print graph funnel id(Authorsandyearofpublication)
metatrim _ES _seES, reffect mixed print graph funnel id(Authorsandyearofpublication)
I used above mentioned codes without and with the code 'mixed'. Results were identical at both stages. The Stata output says no trimming performed; data unchanged. However, the random effect pooled estimate (red font) is different to the original pooled estimate received from metaprop command. What would be the possible reasons? In which occasions trimming is not performed?
Note: default data input format (theta, se_theta) assumed.
Meta-analysis
| Pooled 95% CI Asymptotic No. of
Method | Est Lower Upper z_value p_value studies
-------+----------------------------------------------------
Fixed | 0.218 0.211 0.225 59.430 0.000 68
Random | 0.187 0.153 0.222 10.657 0.000
Test for heterogeneity: Q= 1421.736 on 67 degrees of freedom (p= 0.000)
Moment-based estimate of between studies variance = 0.019
Trimming estimator: Linear
Meta-analysis type: Random-effects model initially, but mixed mode
iteration | estimate Tn # to trim diff
----------+--------------------------------------
1 | 0.218 832 0 2346
2 | 0.218 832 0 0
Note: no trimming performed; data unchanged
Filled
Meta-analysis
| Pooled 95% CI Asymptotic No. of
Method | Est Lower Upper z_value p_value studies
-------+----------------------------------------------------
Fixed | 0.218 0.211 0.225 59.430 0.000 68
Random | 0.187 0.153 0.222 10.657 0.000
Test for heterogeneity: Q= 1421.736 on 67 degrees of freedom (p= 0.000)
Moment-based estimate of between studies variance = 0.019
Step 4- Then I performed metatrim command with the 'flip' option and both 'flip' and 'mixed' options. Then trimming has performed but results are different. I wonder which option to be used in my case (proportion meta-analysis)
metatrim _ES _seES, reffect flip print graph funnel id(Authorsandyearofpublication)
Note: default data input format (theta, se_theta) assumed.
Meta-analysis
| Pooled 95% CI Asymptotic No. of
Method | Est Lower Upper z_value p_value studies
-------+----------------------------------------------------
Fixed | 0.218 0.211 0.225 59.430 0.000 68
Random | 0.187 0.153 0.222 10.657 0.000
Test for heterogeneity: Q= 1421.736 on 67 degrees of freedom (p= 0.000)
Moment-based estimate of between studies variance = 0.019
Trimming estimator: Linear
Meta-analysis type: Random-effects model
iteration | estimate Tn # to trim diff
----------+--------------------------------------
1 | 0.187 1300 4 2346
2 | 0.196 1364 6 128
3 | 0.200 1394 7 60
4 | 0.202 1405 7 22
5 | 0.202 1405 7 0
Filled
Meta-analysis
| Pooled 95% CI Asymptotic No. of
Method | Est Lower Upper z_value p_value studies
-------+----------------------------------------------------
Fixed | 0.234 0.227 0.240 68.002 0.000 75
Random | 0.202 0.170 0.234 12.338 0.000
Test for heterogeneity: Q= 1565.953 on 74 degrees of freedom (p= 0.000)
Moment-based estimate of between studies variance = 0.018
. metatrim _ES _seES, reffect flip mixed print graph funnel id(Authorsandyearofpublication)
Note: default data input format (theta, se_theta) assumed.
Meta-analysis
| Pooled 95% CI Asymptotic No. of
Method | Est Lower Upper z_value p_value studies
-------+----------------------------------------------------
Fixed | 0.218 0.211 0.225 59.430 0.000 68
Random | 0.187 0.153 0.222 10.657 0.000
Test for heterogeneity: Q= 1421.736 on 67 degrees of freedom (p= 0.000)
Moment-based estimate of between studies variance = 0.019
Trimming estimator: Linear
Meta-analysis type: Random-effects model initially, but mixed mode
iteration | estimate Tn # to trim diff
----------+--------------------------------------
1 | 0.218 1514 10 2346
2 | 0.248 1768 18 508
3 | 0.264 1912 22 288
4 | 0.270 1956 23 88
5 | 0.272 1966 23 20
6 | 0.272 1966 23 0
Filled
Meta-analysis
| Pooled 95% CI Asymptotic No. of
Method | Est Lower Upper z_value p_value studies
-------+----------------------------------------------------
Fixed | 0.272 0.266 0.279 83.859 0.000 91
Random | 0.257 0.223 0.291 14.820 0.000
Test for heterogeneity: Q= 2427.665 on 90 degrees of freedom (p= 0.000)
Moment-based estimate of between studies variance = 0.025
I look forward to hearing from you.
Thank you.
Deepani
Comment