Meta-analysis is commonly used in reviews of the effectiveness of medical
technologies, but this approach has not been used in direct support of
guidelines development groups. This paper describes the approach of the North
of England Guidelines Development Project in describing the evidence using
meta-analyses that were conducted explicitly to address questions on the
choice of therapy raised by the guidelines development groups. Particular
emphasis is placed on the context within which the contributing trials were
conducted and the extent to which systematic differences between trials
(heterogeneity) was observed, described, and explained. There is a trade-off
between internal and external validity for different metrics when presenting
the results of trials. More interpretable metrics, such as risk differences or
weighted mean differences, are confounded by study design issues and strong
assumptions. More robust measures such as odds ratios or standardized weighted
mean differences are difficult to interpret physically. Individual patient
data may prove particularly helpful in addressing pivotal questions on the
magnitude of effects of interventions, though accessing and reanalyzing these
data requires a substantial investment in time and other resources.