Correct weighting in meta-analysis

[David Ralf Simon]

Hi everyone!

Recently I conducted two separate meta-analyses (956 and 544 observations) for two different but related topics, both with categorical outcome variables.
In the next step I merged both datasets to analyze how the results of one meta-analysis affects the results of the other. I matched every observation where the country of the observed paper is the same and the observation period is roughly equal as well (+/- 3 years for first and last year).
This is the first time I had to apply the M:M merge, as I wanted to find every possible match between the data sets. As a result I have a data set with now 1956 observations, as an observation of one meta analysis matched with up to 44 observations of the other. However, others did not match at all or only match once, or twice for example.
This is why I want to apply weighting to my meta-regression so that the observations of my new data set that are based on underlying observations that were matched more often, "count less".
Optimally I would also like to give less weight to those observations where the years did not match exactly. However if I understand correctly I can only weight using one criteria.
This means I would have to create some kind of index that includes both how often the underlying observation was matched and also how exact the observation periods matched?
My question would then be which weight to use in Stata as this is a pretty unique problem that does not fit to any of the standard examples of pweight, fweigt, aweigth and iweight.

I hope my issue is somewhat understandable!

[Tiago Pereira]

You explanation started well, but somehow I lost you after the first sentence.

[David Ralf Simon]

I can try to clarify it.
In my one meta-analysis (which looks at the Energy-Growth Nexus) an observation could be:
Outcome: "Growth hypothesis"
Country: China
Observation period: 1970-2012

In the other one (which looks at the Environmental Kuznets Curve (EKC)) an observation could be:
Outcome: "EKC holds"
Country: China
Observation period: 1971-2014

These two I would match, as the country is the same and the observation period is roughly equal when I apply my rule mentioned above. I merge these two data sets because I want to find out whether there is any relation between the Energy-Growth Nexus and the Environmental Kuznets Curve.

Nevertheless, these two observations could also be matched with several other observations from the respective other data set.
In the end, I want to apply weighting, as if for instance the first example observation gets matched to let's say 15 other observations of the EKC-meta-analysis, while another observations whose result is "Conservation hypothesis" gets matched only five times. In my opinion, this would lead to biased results, as in the original data set both "Growth hypothesis" and "Conservation hypothesis" appear once, in my new merged data set "Growth hypothesis" would appear 15 times, while "Conservation hypothesis" only 5 times. I am however not sure how to apply this weight.

[David Ralf Simon]

As I mentioned above, ideally I would also account for the difference in observation periods. In the example merge above, the difference would be (1971-1970)+(2014-2012) = 3.
In another example, let's say 1980-2010 and 1981-2010 the difference would be only 1. If possible I would also like to give more weight to those observations where the data fits better.
However, this is less important than accounting for the times an underlying observation was included.

[David Fisher]

Dear David,

I cannot offer much advice, as like Tiago I find it hard to follow the details. But have you considered making use of multivariate meta-analysis? This is a technique which allows multiple outcomes to be meta-analysed within the same model, with correlation structures that allow "borrowing of strength" from correlated observations: that is, observations relating to the same studies but different outcomes. If I understand correctly, you do not formally have a set of studies with multiple outcomes, but rather sets of data which you would like to match (on country and observation period). Therefore, you may wish to adjust (and/or weight) somehow for the closeness of the match. I am not sure how best to accomplish this, but possibly it may have already been considered somewhere in the literature.

I hope that is of some help. But this does sound like quite an unusual study, and you may find it difficult to get definitive answers.

Best wishes,

David.