Review ArticleUncertainty method improved on best–worst case analysis in a binary meta-analysis
Section snippets
Handling missing binary data in a meta-analysis
The intention-to-treat approach is generally recommended to minimize bias in the design, conduct, and analysis of randomized controlled trials of effectiveness [1]. Most systematic reviewers aim to perform an intention-to-treat meta-analysis, including all randomized participants from each trial, but in practice this is not always straightforward. It is often apparent in published reports of randomized controlled trials that a number of participants have not been included in the analysis, even
Best–worst case analysis
Figure 1a shows a meta-analysis from a review comparing the effectiveness of two antimalarial drug treatments, artemether-lumefantrine (AL) and mefloquine plus artesunate (MA) [6]. The complete-case analysis is shown for each of the three trials (Lefevre 2001, van Vugt 2000, and van Vugt 1998; cited in [6]), along with a pooled estimate and confidence interval calculated using Woolf's inverse variance method [7]. The percentage of subjects with missing data within each trial is also shown.
Allowing for uncertainty due to missing binary data
In the context of missing survey responses, Molenberghs et al. [9] have described uncertainty as arising not only from imprecision due to sampling error, but also from ignorance due to incomplete data. They suggest the use of a region of uncertainty—a larger region, in the spirit of a confidence region but designed to capture the combined effects of imprecision and ignorance. A natural estimate of the region of uncertainty is the union of confidence regions for an estimated effect across a full
Simulation study
Using simulation, we investigated the attributes of the uncertainty method in comparison to complete-case and best–worst case analysis, in situations with and without an underlying treatment effect, and with informative and noninformative missing data. (Informative missing data occurs when observation of the outcome is dependent upon its value.)
We considered the scenario of a meta-analysis of three trials, each with the same sample size, and with an event rate of 0.25 in the control arm.
Simulation results
We first consider the situation with no underlying treatment effect (OR = 1) and informative missing data generated using MOR = 2 (Table 3). The full data analysis consistently provides close to 95% coverage, as expected, because there are no missing data in this analysis. Coverage of the complete-case method is close to 95% with small sample size or few missing data, but decreases as sample size and the proportion of missing data increase, indicating increasing bias. With 150 participants per arm
Discussion
The intention-to-treat principle is usually advocated for individual clinical trials. If some outcomes are unknown, no method of analysis can provide a pure intention-to-treat estimate. There has been much research and debate on the handling of missing data in an intention-to-treat analysis [1], [2]. Preferred approaches can be identified for some specific situations [2], [10], [11], [12]. For example, if the missing data can be assumed to be noninformative, then complete-case analysis will be
References (20)
- et al.
Intention-to-treat approach to data from randomized controlled trials: a sensitivity analysis
J Clin Epidemiol
(2003) Meta-analysis adjusting for compliance: the example of screening for breast cancer
J Clin Epidemiol
(1992)- et al.
What is meant by intention to treat analysis? Survey of published randomised controlled trials
BMJ
(1999) Problems in dealing with missing data and informative censoring in clinical trials
Curr Control Trials Cardiovasc Med
(2002)- et al.
Systematic reviews in healthcare: meta-analysis in context
(2001) - et al.
Evidence-based medicine: how to practice and teach EBM
(1997) - et al.
Analysing and presenting results
- et al.
Artemether-lumefantrine for treating uncomplicated falciparum malaria
On estimating the relation between blood group and disease
Ann Hum Genet
(1955)A graphical sensitivity analysis for clinical trials with non-ignorable missing binary outcome
Stat Med
(2002)