Meta-analysis: Overview

Professor Andy Field

University of Sussex

What is a meta-analysis?

Glass (1976)

An “analysis of analyses”

Aim

Combine, summarize and interpret all available evidence pertaining to a clearly defined research field or research question

  • Estimate the ‘true’ effect (and uncertainty around it) by combining results from research that addresses the same question.
  • Estimate variability in effects across studies
  • Estimate predictors of effect sizes (so called meta-regression)

Some pitfalls

  • Comparing apples and oranges
    • Make sure the research question is clearly defined
    • Have consistent inclusion criteria
  • The file-drawer problem
    • Contact authors for null results
    • Selection bias models
  • Research agenda bias
    • Have objective inclusion criteria

Six steps in a meta-analysis

  1. Identify a clearly defined research question
  2. Literature search
  3. Apply inclusion criteria
  4. Extract data and calculate effect sizes
  5. Fit the model
  6. Write-it up

Six steps in a meta-analysis

  1. Identify a clearly defined research question
  2. Literature search
  3. Apply inclusion criteria
  4. Extract data and calculate effect sizes
  5. Fit the model
  6. Write-it up

Flack, Z. M., Field, A. P., & Horst, J. S. (2018). The effects of shared storybook reading on word learning: A meta-analysis. Developmental Psychology, 54(7), 1334–1346. doi: 10.1037/dev0000512

Six steps in a meta-analysis

  1. Identify a clearly defined research question
  2. Literature search
  3. Apply inclusion criteria
  4. Extract data and calculate effect sizes
  5. Fit the model
  6. Write-it up

Examples of inclusion criteria

Sources of bias

  • Comparing apples and oranges
  • GIGO
  • Measures:
    • What measures are considered methodologically credible/valid/reliable
  • Participants
    • Specific populations or general?
  • Controls
    • What is an appropriate control group
  • Study design
    • What is the gold standard design - RCT?

Essential resources