Meta-Analysis Overview

What is Meta-Analysis?

Meta-analysis is a statistical technique that combines results from multiple studies to produce a single summary estimate. It increases statistical power and precision beyond individual studies and can resolve inconsistencies across studies.

When to Use Meta-Analysis

  • You have 2+ studies measuring the same outcome
  • Studies are sufficiently similar (population, intervention, design)
  • Outcome data is reported in combinable format
  • Heterogeneity is not prohibitively high
  • Pooling makes clinical/scientific sense

When NOT to Meta-Analyze

  • Only one study available
  • Studies too heterogeneous (different populations, interventions, outcomes)
  • Poor quality studies that would bias results
  • Insufficient data reported for effect size calculation
  • Pooling wouldn't answer a meaningful question

Remember: Just because you can meta-analyze doesn't mean you should. Narrative synthesis may be more appropriate for highly heterogeneous studies.

Types of Meta-Analysis

Fixed-Effect Model

Assumes all studies estimate the same underlying effect:

  • Use when: Studies are very similar, heterogeneity is low
  • Gives more weight to larger studies
  • Narrower confidence intervals
  • Less commonly used (most reviews have some heterogeneity)

Random-Effects Model

Assumes studies estimate different but related effects:

  • Use when: Some heterogeneity expected (most cases)
  • Accounts for between-study variance
  • Wider confidence intervals (more conservative)
  • Generally preferred for systematic reviews

Effect Size Metrics

For Continuous Outcomes

Standardized Mean Difference (SMD):

  • Use when: Different scales measuring same construct
  • Example: Various depression scales (BDI, HAM-D, PHQ-9)
  • Expressed in standard deviation units
  • Cohen's d interpretation: 0.2=small, 0.5=medium, 0.8=large

Mean Difference (MD):

  • Use when: All studies use same scale
  • Example: All studies report HbA1c in %
  • Results in original units (clinically interpretable)
  • Preferred when possible

For Binary Outcomes

Risk Ratio (RR) / Relative Risk:

  • Ratio of event probability in intervention vs control
  • Easy to interpret
  • RR = 1 means the event is equally likely in both groups
  • RR < 1 means the event is less likely in the intervention group; RR > 1 means it is more likely
  • Whether either direction is favorable depends on the outcome and how the groups are ordered

Odds Ratio (OR):

  • Ratio of odds of event in intervention vs control
  • Useful for case-control studies
  • Less intuitive than RR
  • Similar to RR when events are rare (<10%)

Risk Difference (RD):

  • Absolute difference in event rates
  • Most clinically interpretable
  • Example: RD = -0.10 means 10% absolute risk reduction

Meta-Analysis in Scholara

  1. 1Ensure outcome data has been extracted via the Analysis Chat (or ask the AI to extract it)
  2. 2In the Analysis Chat, ask the AI to run a meta-analysis (e.g., 'Run a random-effects meta-analysis for depression outcomes')
  3. 3The AI extracts the needed data, selects the appropriate effect size metric, and runs the analysis
  4. 4A forest plot and summary statistics appear in the chat as interactive assets
  5. 5Refine by asking follow-up questions (e.g., 'Use fixed-effect model instead', 'Add subgroup analysis by study design')
  6. 6Export the forest plot and results table in your preferred format

Interpreting Results

Summary Effect

  • Point estimate: Average effect across studies
  • 95% CI: Range of plausible values
  • If CI excludes null (0 for MD/SMD, 1 for RR/OR), effect is statistically significant
  • p-value: Assuming the null hypothesis and statistical model are correct, the probability of results at least as extreme as those observed; it does not measure clinical importance

Heterogeneity Statistics

I² (I-squared):

  • % of variability due to heterogeneity vs chance
  • 0-40%: Low heterogeneity
  • 30-60%: Moderate heterogeneity
  • 50-90%: Substantial heterogeneity
  • 75-100%: Considerable heterogeneity

Tau² (Tau-squared):

  • Variance of true effects across studies
  • Used in random-effects model
  • Larger values = more heterogeneity

Cochran's Q test:

  • Tests null hypothesis of homogeneity
  • p < 0.10 suggests significant heterogeneity
  • Note: Low power with few studies, high power with many

Dealing with Heterogeneity

If heterogeneity is high (I² > 50%), consider:

  1. 1Check for data entry errors
  2. 2Investigate sources (subgroup analysis, meta-regression)
  3. 3Sensitivity analysis excluding outliers
  4. 4Use random-effects model (already accounts for heterogeneity)
  5. 5Consider narrative synthesis instead of pooling
  6. 6Report heterogeneity and discuss limitations

Publication Bias

Small, non-significant studies are less likely to be published, potentially biasing results. Scholara provides tools to assess:

  • Funnel plot: Visual inspection for asymmetry
  • Egger's test: Statistical test for funnel plot asymmetry
  • Trim-and-fill: Estimates effect after adjusting for missing studies
  • Fail-safe N: How many null studies needed to change conclusion

Quality of Evidence

Use GRADE approach to rate certainty of evidence:

  • High: Very confident in estimate
  • Moderate: Moderately confident; true effect likely close
  • Low: Limited confidence; true effect may differ substantially
  • Very low: Very little confidence; true effect likely substantially different
  • Downgrade for: risk of bias, inconsistency, indirectness, imprecision, publication bias

Meta-analysis provides a precise summary, but precision ≠ accuracy. Always interpret results in context of study quality, heterogeneity, and potential biases. A well-done narrative synthesis is better than a poorly done meta-analysis.

References