META -ANALYSES negative value of a confidence interval - How to explain???

[Tom Salvitti]

Hi to everybody,

I use the command metaprop (My effect size is "Prevalence of a skin disease")

Study	Prevalence of a skin disease	95%ic	95%ic	%weight
1	0.032	0.001	0.063	1.20
2	0.087	0.006	0.168	0.19
3	0.015	0.002	0.029	4.69
4	0.012	-0.012	0.036	1.87
5	0.029	-0.027	0.086	0.38
6	0.006	-0.005	0.017	5.88
7	0.005	-0.005	0.016	6.25
8	0.063	-0.006	0.131	0.26
9	0.002	0.000	0.004	13.36
10	0.0018	0.001	0.036	3.15
11	0.013	-0.012	0.037	1.79
12	0.003	0.002	0.005	13.58
13	0.014	0.011	0.017	12.45
14	0.002	0.001	0.003	13.71
15	0.022	-0.008	0.052	1.28
16	0.001	-0.000	0.002	13.59
17	0.005	-0.005	0.015	6.40

My question is: How is it possible, How can explain that in some studies the lower bound of the confidence interval is negative? I wonder this because the effect size is a prevalence.
Thank you all!!!

Tommaso

[Nick Cox]

The recipe confidence interval = estimate +/- multiple of standard error is not a good one for small proportions. Study 4 gives the game away: the confidence interval is, to the resolution shown, 0.012 +/- 0.024 and the lower bound is therefore negative.

I don't know anything about metaprop (which is from SSC, as you are asked to explain), but it may offer handles for this problem.

[Tom Salvitti]

Thanks a lot..In fact the proportion is very small. I understand...I thought it was my mistake

[Nick Cox]

A reviewer might think it a mistake to use a method that isn't suited for the purpose. There's been discussion of confidence intervals for proportions over several decades in many journals.

[Tom Salvitti]

Yes but, I have to use this method i try to a meta regress. I have the effect size measure in the data set (have precalculated effect sizes,).
If you do not have the summary data from individual studies and, instead, you have precalculated effect sizes, you can use meta set to declare your meta-analysis data (help meta esize)

So i think it's not a mistake but it's due to small proportions


Thanks a lot!!!

[Nick Cox]

I would like to hear from experts in meta analysis about #5. Confidence intervals that don't make complete sense don't do anybody -- researchers or their readerships -- any good. (And truncating the intervals at zero isn't a good solution either.)

[Tom Salvitti]

Me too, but nobody answer and I try to solve my problem alone...

[Tom Salvitti]

And I don't think that in this forum there is nobody expert in meta analysis


However thanks a lot Nick Cox

[Nick Cox]

No; that is quite wrong. There are major experts on meta analysis who are members here, but the people I can think of visit only irregularly. I don't think it's especially polite to ping them, however.

[Tom Salvitti]

if unintentionally I have not been polite I apologize but this is not the first time that I have asked for advice on meta analysis and no one responds. I repeat I apologize if I turned out to be unpolite.

[Nick Cox]

I don't think you're being impolite at all. We are both correct: questions on meta often don't get answered here but major meta experts do visit occasionally.

Back to basics: Statalist is based on volunteering. That works well when people get advice that is good and quick and free -- and not so well otherwise.

[Leonardo Guizzetti]

The data you have appears to use a Wald-type interval on the raw proportion scale, and it would be much better to compute the point estimate and standard errors (and intervals) using a reasonable scale (e.g., logit) which ensure that when the values are very close to 0 or 1, the interval will still be entirely inside the range of [0, 1]. Unfortunately, I don't think -metaprop- did this calculation for you, and it seems limited by not having a logit transformation. Internally, it is truncating the intervals, which is cosmetically covering up this deficiency. In this day and age with the plethora of meta-analysis software available, a reviewer would be right to criticize these data as presented.

Do you have the raw data to compute the prevalence? If so, I would use -metan- (SSC; now maintained by David Fisher) to compute meta-analysis of single proportions, or else use the -meta- suite that comes with Stata 17, though that requires greater technical understanding for single proportions.

[Tom Salvitti]

Good morning to everybody

Thanks Leonardo Guizzetti

My original dataset is:

Study	Total Diabetics	Diabetics with skin disease
1	125	4
2	46	4
3	328	5
4	81	1
5	34	1
6	177	1
7	187	1
8	48	3
9	2568	5
10	222	4
11	79	1
12	9105	31
13	5276	72
14	12271	22
15	92	2
16	2352	2
17	191	1

my effect size is prevalence Diabetics with skin disease...
I use stata 16.
how would you proceed? Thanks a lot in advance

[Joseph Coveney]

I use stata 16.
how would you proceed?

Maybe something like this?

.ÿ
.ÿversionÿ16.1

.ÿ
.ÿclearÿ*

.ÿ
.ÿquietlyÿinputÿbyteÿsidÿintÿtotÿbyteÿpos

.ÿ
.ÿglmÿposÿi.sid,ÿfamily(binomialÿtot)ÿlink(logit)ÿnolog

GeneralizedÿlinearÿmodelsÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿNumberÿofÿobsÿÿÿ=ÿÿÿÿÿÿÿÿÿ17
Optimizationÿÿÿÿÿ:ÿMLÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿResidualÿdfÿÿÿÿÿ=ÿÿÿÿÿÿÿÿÿÿ0
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿScaleÿparameterÿ=ÿÿÿÿÿÿÿÿÿÿ1
Devianceÿÿÿÿÿÿÿÿÿ=ÿÿ2.46470e-14ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ(1/df)ÿDevianceÿ=ÿÿÿÿÿÿÿÿÿÿ.
Pearsonÿÿÿÿÿÿÿÿÿÿ=ÿÿ6.22702e-23ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ(1/df)ÿPearsonÿÿ=ÿÿÿÿÿÿÿÿÿÿ.

Varianceÿfunction:ÿV(u)ÿ=ÿu*(1-u/tot)ÿÿÿÿÿÿÿÿÿÿÿÿÿ[Binomial]
Linkÿfunctionÿÿÿÿ:ÿg(u)ÿ=ÿln(u/(tot-u))ÿÿÿÿÿÿÿÿÿÿÿ[Logit]

ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿAICÿÿÿÿÿÿÿÿÿÿÿÿÿ=ÿÿÿ5.116037
Logÿlikelihoodÿÿÿ=ÿ-26.48631154ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿBICÿÿÿÿÿÿÿÿÿÿÿÿÿ=ÿÿÿ2.46e-14

------------------------------------------------------------------------------
ÿÿÿÿÿÿÿÿÿÿÿÿÿ|ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿOIM
ÿÿÿÿÿÿÿÿÿposÿ|ÿCoefficientÿÿstd.ÿerr.ÿÿÿÿÿÿzÿÿÿÿP>|z|ÿÿÿÿÿ[95%ÿconf.ÿinterval]
-------------+----------------------------------------------------------------
ÿÿÿÿÿÿÿÿÿsidÿ|
ÿÿÿÿÿÿÿÿÿÿ2ÿÿ|ÿÿÿ1.058121ÿÿÿÿ.729434ÿÿÿÿÿ1.45ÿÿÿ0.147ÿÿÿÿ-.3715435ÿÿÿÿ2.487785
ÿÿÿÿÿÿÿÿÿÿ3ÿÿ|ÿÿ-.7587182ÿÿÿ.6792352ÿÿÿÿ-1.12ÿÿÿ0.264ÿÿÿÿ-2.089995ÿÿÿÿ.5725583
ÿÿÿÿÿÿÿÿÿÿ4ÿÿ|ÿÿ-.9725305ÿÿÿ1.127282ÿÿÿÿ-0.86ÿÿÿ0.388ÿÿÿÿ-3.181962ÿÿÿÿ1.236901
ÿÿÿÿÿÿÿÿÿÿ5ÿÿ|ÿÿ-.0870114ÿÿÿ1.135151ÿÿÿÿ-0.08ÿÿÿ0.939ÿÿÿÿ-2.311866ÿÿÿÿ2.137843
ÿÿÿÿÿÿÿÿÿÿ6ÿÿ|ÿÿ-1.760988ÿÿÿ1.124254ÿÿÿÿ-1.57ÿÿÿ0.117ÿÿÿÿ-3.964484ÿÿÿÿ.4425089
ÿÿÿÿÿÿÿÿÿÿ7ÿÿ|ÿÿÿ-1.81625ÿÿÿ1.124118ÿÿÿÿ-1.62ÿÿÿ0.106ÿÿÿÿ-4.019481ÿÿÿÿ.3869799
ÿÿÿÿÿÿÿÿÿÿ8ÿÿ|ÿÿÿÿ.701446ÿÿÿ.7834667ÿÿÿÿÿ0.90ÿÿÿ0.371ÿÿÿÿ-.8341205ÿÿÿÿ2.237012
ÿÿÿÿÿÿÿÿÿÿ9ÿÿ|ÿÿÿÿÿÿ-2.83ÿÿÿ.6772405ÿÿÿÿ-4.18ÿÿÿ0.000ÿÿÿÿ-4.157367ÿÿÿ-1.502633
ÿÿÿÿÿÿÿÿÿ10ÿÿ|ÿÿ-.5887045ÿÿÿ.7161366ÿÿÿÿ-0.82ÿÿÿ0.411ÿÿÿÿ-1.992306ÿÿÿÿ.8148974
ÿÿÿÿÿÿÿÿÿ11ÿÿ|ÿÿ-.9472126ÿÿÿ1.127424ÿÿÿÿ-0.84ÿÿÿ0.401ÿÿÿÿ-3.156923ÿÿÿÿ1.262498
ÿÿÿÿÿÿÿÿÿ12ÿÿ|ÿÿ-2.269685ÿÿÿ.5391036ÿÿÿÿ-4.21ÿÿÿ0.000ÿÿÿÿ-3.326309ÿÿÿ-1.213061
ÿÿÿÿÿÿÿÿÿ13ÿÿ|ÿÿ-.8710205ÿÿÿ.5218673ÿÿÿÿ-1.67ÿÿÿ0.095ÿÿÿÿ-1.893862ÿÿÿÿ.1518206
ÿÿÿÿÿÿÿÿÿ14ÿÿ|ÿÿ-2.912661ÿÿÿ.5511811ÿÿÿÿ-5.28ÿÿÿ0.000ÿÿÿÿ-3.992956ÿÿÿ-1.832366
ÿÿÿÿÿÿÿÿÿ15ÿÿ|ÿÿ-.3971663ÿÿÿ.8771406ÿÿÿÿ-0.45ÿÿÿ0.651ÿÿÿÿÿ-2.11633ÿÿÿÿ1.321998
ÿÿÿÿÿÿÿÿÿ16ÿÿ|ÿÿ-3.659527ÿÿÿ.8710281ÿÿÿÿ-4.20ÿÿÿ0.000ÿÿÿÿ-5.366711ÿÿÿ-1.952343
ÿÿÿÿÿÿÿÿÿ17ÿÿ|ÿÿ-1.837528ÿÿÿ1.124067ÿÿÿÿ-1.63ÿÿÿ0.102ÿÿÿÿÿ-4.04066ÿÿÿÿ.3656038
ÿÿÿÿÿÿÿÿÿÿÿÿÿ|
ÿÿÿÿÿÿÿ_consÿ|ÿÿ-3.409496ÿÿÿ.5081973ÿÿÿÿ-6.71ÿÿÿ0.000ÿÿÿÿ-4.405545ÿÿÿ-2.413448
------------------------------------------------------------------------------

.ÿ
.ÿpredictÿdoubleÿxb,ÿxb

.ÿpredictÿdoubleÿstdp,ÿstdp

.ÿ
.ÿgenerateÿdoubleÿlbÿ=ÿinvlogit(xbÿ+ÿinvnormal(0.05)ÿ*ÿstdp)

.ÿgenerateÿdoubleÿubÿ=ÿinvlogit(xbÿ+ÿinvnormal(0.95)ÿ*ÿstdp)

.ÿ
.ÿgenerateÿdoubleÿprÿ=ÿinvlogit(xb)

.ÿformatÿprÿlbÿubÿ%05.3f

.ÿ
.ÿlistÿsidÿtotÿposÿprÿlbÿub,ÿnoobsÿseparator(0)

ÿÿ+-------------------------------------------+
ÿÿ|ÿsidÿÿÿÿÿtotÿÿÿposÿÿÿÿÿÿprÿÿÿÿÿÿlbÿÿÿÿÿÿubÿ|
ÿÿ|-------------------------------------------|
ÿÿ|ÿÿÿ1ÿÿÿÿÿ125ÿÿÿÿÿ4ÿÿÿ0.032ÿÿÿ0.014ÿÿÿ0.071ÿ|
ÿÿ|ÿÿÿ2ÿÿÿÿÿÿ46ÿÿÿÿÿ4ÿÿÿ0.087ÿÿÿ0.039ÿÿÿ0.184ÿ|
ÿÿ|ÿÿÿ3ÿÿÿÿÿ328ÿÿÿÿÿ5ÿÿÿ0.015ÿÿÿ0.007ÿÿÿ0.031ÿ|
ÿÿ|ÿÿÿ4ÿÿÿÿÿÿ81ÿÿÿÿÿ1ÿÿÿ0.012ÿÿÿ0.002ÿÿÿ0.061ÿ|
ÿÿ|ÿÿÿ5ÿÿÿÿÿÿ34ÿÿÿÿÿ1ÿÿÿ0.029ÿÿÿ0.006ÿÿÿ0.139ÿ|
ÿÿ|ÿÿÿ6ÿÿÿÿÿ177ÿÿÿÿÿ1ÿÿÿ0.006ÿÿÿ0.001ÿÿÿ0.029ÿ|
ÿÿ|ÿÿÿ7ÿÿÿÿÿ187ÿÿÿÿÿ1ÿÿÿ0.005ÿÿÿ0.001ÿÿÿ0.027ÿ|
ÿÿ|ÿÿÿ8ÿÿÿÿÿÿ48ÿÿÿÿÿ3ÿÿÿ0.063ÿÿÿ0.024ÿÿÿ0.151ÿ|
ÿÿ|ÿÿÿ9ÿÿÿÿ2568ÿÿÿÿÿ5ÿÿÿ0.002ÿÿÿ0.001ÿÿÿ0.004ÿ|
ÿÿ|ÿÿ10ÿÿÿÿÿ222ÿÿÿÿÿ4ÿÿÿ0.018ÿÿÿ0.008ÿÿÿ0.040ÿ|
ÿÿ|ÿÿ11ÿÿÿÿÿÿ79ÿÿÿÿÿ1ÿÿÿ0.013ÿÿÿ0.002ÿÿÿ0.063ÿ|
ÿÿ|ÿÿ12ÿÿÿÿ9105ÿÿÿÿ31ÿÿÿ0.003ÿÿÿ0.003ÿÿÿ0.005ÿ|
ÿÿ|ÿÿ13ÿÿÿÿ5276ÿÿÿÿ72ÿÿÿ0.014ÿÿÿ0.011ÿÿÿ0.017ÿ|
ÿÿ|ÿÿ14ÿÿÿ12271ÿÿÿÿ22ÿÿÿ0.002ÿÿÿ0.001ÿÿÿ0.003ÿ|
ÿÿ|ÿÿ15ÿÿÿÿÿÿ92ÿÿÿÿÿ2ÿÿÿ0.022ÿÿÿ0.007ÿÿÿ0.067ÿ|
ÿÿ|ÿÿ16ÿÿÿÿ2352ÿÿÿÿÿ2ÿÿÿ0.001ÿÿÿ0.000ÿÿÿ0.003ÿ|
ÿÿ|ÿÿ17ÿÿÿÿÿ191ÿÿÿÿÿ1ÿÿÿ0.005ÿÿÿ0.001ÿÿÿ0.027ÿ|
ÿÿ+-------------------------------------------+

.ÿ
.ÿmarginsÿ,ÿexpression(invlogit(predict(xb)))

PredictiveÿmarginsÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿNumberÿofÿobsÿ=ÿ17
ModelÿVCE:ÿOIM

Expression:ÿinvlogit(predict(xb))

------------------------------------------------------------------------------
ÿÿÿÿÿÿÿÿÿÿÿÿÿ|ÿÿÿÿÿÿÿÿÿÿÿÿDelta-method
ÿÿÿÿÿÿÿÿÿÿÿÿÿ|ÿÿÿÿÿMarginÿÿÿstd.ÿerr.ÿÿÿÿÿÿzÿÿÿÿP>|z|ÿÿÿÿÿ[95%ÿconf.ÿinterval]
-------------+----------------------------------------------------------------
ÿÿÿÿÿÿÿ_consÿ|ÿÿÿ.0193381ÿÿÿ.0040716ÿÿÿÿÿ4.75ÿÿÿ0.000ÿÿÿÿÿ.0113579ÿÿÿÿ.0273184
------------------------------------------------------------------------------

.ÿ
.ÿexit

endÿofÿdo-file


.

[Tom Salvitti]

thanks a lot!!!