Meta-analysis is an established statistical technique closely associated with systematic reviews of the literature. The concepts of a statistical meta-analysis are increasingly applied to pharmacometric analysis (model-based meta-analysis). The common feature of both approaches is that the experimental “unit” is a publication and they seek to address the problem of small sample sizes. The Central Limit Theorem states that for “n” samples from a population distribution, the precision of the sample mean is proportional to 1/sqrt(n). Studies with small sample sizes are therefore often confounded by poor estimates of population data (and undue influence of outliers). The components of statistical meta-analysis are: 1. A systematic review of the literature using pre-determined search and quality criteria. 2. Identification of a summary statistic that can be pooled across the publications (classically an odds ratio). 3. A regression style analysis to pool the summary statistic across the studies, often presented as a “forest plot” where the effect size and its confidence interval are compared graphically to the reference value of zero (no effect). 4. An assessment of publication bias (often using a funnel plot) which looks for censoring of the pooled effect size data (e.g. an absence of negative studies from the literature). The archetypical statistical meta-analysis was of the use of pre-term maternal corticosteroid administration to promote lung development in premature infants (1). It has been claimed that if this meta-analysis was done at the time of the original trials, the treatment would have been adopted a decade earlier and many lives saved. The forest plot for this analysis is used for the logo of the Cochran Collaboration. While a recent subsequent meta-analysis confirmed the findings of the original meta-analysis, the attendant correspondence with its publication highlight some of the limitations of statistical meta-analysis that could be supplemented by a model-based analysis. There are (at least) two types of model based meta-analysis. First, a meta-analysis can be used to assess inter-study variability when there are a number of published population pharmacokinetic studies for a particular drug. The parameters for each model are extracted from the published papers and used to simulate the time-course of population predicted concentrations for each study using a standard dose regimen and sample schedule. These are compiled in a new database, with study “id” becoming the population unit. The objective function is weighted for the number of subjects in each of the original studies. Fitting the model gives estimates of a “grand” population pharmacokinetic parameters, and their variability terms reflect inter-study variability. Simulations of predicted concentrations and their 95% confidence intervals can be used to look for outlying studies. There are sufficient published population pharmacokinetic studies of docetaxel, an anti-neoplatic drug with high inter-subject and inter-study variability in kinetics, to use this approach. The second type of model-based meta-analysis is the development of a unifying model based on a number of heterogeneous published studies. Software such as digitizeit (www.digitizeit.de) conveniently allow numerical data to be extracted from published graphs. Such a unifying model has been successfully developed for the intravenous anaesthetic propofol, where separate published studies of the cerebral, lung and systemic kinetics of propofol were integrated into a single physiologically based recirculatory model with acceptable predictive performance (2). This type of meta-modelling analysis is a very rigorous quantitative test of a knowledge base that is often only described qualitatively. A similar attempt to develop a model of the effects of propofol on the cerebrovascular system was thwarted by a lack of quality published data to support established dogma. Meta-analysis has evolved as a device to overcome the limitations of publications based on small sample sizes. However, the need for meta-analysis has in part arisen because of the current system of funding and publishing medical research. The drive for “novel” projects means that studies that simply increase “n” or revisit established knowledge are difficult to fund and publish despite their potential importance and overall contribution. The quality of publications are often assessed by their journal impact factor, but it is very rare for the data underlying the publication to be available to either the reviewers or readers of the journal for reanalysis or pooling with other studies. The role of meta-analysis will no doubt evolve as these challenges are addressed and systems for storing and sharing data develop and expand.
References:
- Crowley P, Chalmers I, Keirse MJNC. The effects of corticosteroid administration before preterm delivery: an overview of the evidence from controlled trials. British Journal of Obstetrics and Gynaecology1990;97:11–25.
- Masui K, Upton RN, Doufas AG, Coetzee JF, Kazama T, Mortier EP, Struys MM. The performance of compartmental and physiologically based recirculatory pharmacokinetic models for propofol: a comparison using bolus, continuous, and target-controlled infusion data. Anesth Analg. 2010 Aug;111(2):368-79.