Is there an excess of significant findings in published studies of psychotherapy for depression?

Research output: Journal contributionsJournal articlesResearchpeer-review

Authors

  • J. Flint
  • Pim Cuijpers
  • J. Horder
  • Sander L. Koole
  • M. R. Munafò
BACKGROUND: Many studies have examined the efficacy of psychotherapy for major depressive disorder (MDD) but publication bias against null results may exist in this literature. However, to date, the presence of an excess of significant findings in this literature has not been explicitly tested.

METHOD: We used a database of 1344 articles on the psychological treatment of depression, identified through systematic search in PubMed, PsycINFO, EMBASE and the Cochrane database of randomized trials. From these we identified 149 studies eligible for inclusion that provided 212 comparisons. We tested for an excess of significant findings using the method developed by Ioannidis and Trikalinos (2007), and compared the distribution of p values in this literature with the distribution in the antidepressant literature, where publication bias is known to be operating.

RESULTS: The average statistical power to detect the effect size indicated by the meta-analysis was 49%. A total of 123 comparisons (58%) reported a statistically significant difference between treatment and control groups, but on the basis of the average power observed, we would only have expected 104 (i.e. 49%) to do so. There was therefore evidence of an excess of significance in this literature (p = 0.010). Similar results were obtained when these analyses were restricted to studies including a cognitive behavioural therapy (CBT) arm. Finally, the distribution of p values for psychotherapy studies resembled that for published antidepressant studies, where publication bias against null results has already been established.

CONCLUSIONS: The small average size of individual psychotherapy studies is only sufficient to detect large effects. Our results indicate an excess of significant findings relative to what would be expected, given the average statistical power of studies of psychotherapy for major depression.
Original languageEnglish
JournalPsychological Medicine
Volume45
Issue number2
Pages (from-to)439-446
Number of pages8
ISSN0033-2917
DOIs
Publication statusPublished - 12.01.2015

    Research areas

  • Psychology
  • Health sciences
  • Cognitive behavioural therapy, depression, excess of significance, meta-Analysis, psychotherapy, publication bias

Recently viewed

Publications

  1. A piezo servo hydraulic actuator for use in camless combustion engines and its control with MPC
  2. A discrete approximate solution for the asymptotic tracking problem in affine nonlinear systems
  3. A duty-block network approach for an integrated driver rostering problem in public bus transport
  4. Effects of elevated atmospheric CO2 concentrations on barley, sugar beet and wheat in a rotation
  5. Choosing between Internet-based psychodynamic versus cognitive behavioral therapy for depression
  6. A geometric approach for the model parameter estimation in a permanent magnet synchronous motor
  7. Effects of global climate change on geographic distributions of vertebrates in North Queensland
  8. The priority value of scrubland habitats for carnivore conservation in Mediterranean ecosystems
  9. Managing increasing environmental risks through agrobiodiversity and agrienvironmental policies
  10. Evolutionary clustering of Lagrangian trajectories in turbulent Rayleigh-Bénard convection flows
  11. Microstructure and mechanical properties of as-cast Mg-Sn-Ca alloys and effect of alloying elements
  12. New methods for the analysis of links between international firm activities and firm performance
  13. On the role of linguistic features for comprehension and learning from STEM texts. A meta-analysis
  14. WHICH ESTIMATION SITUATIONS ARE RELEVANT FOR A VALID ASSESSMENT OF MEASUREMENT ESTIMATION SKILLS
  15. Current and New Research Perspectives on Dynamic Facial Emotion Detection in Emotional Interface
  16. Nonylphenol polyethoxylate degradation by means of electrocoagulation and electrochemical Fenton
  17. A structural property of the wavelet packet transform method to localise incoherency of a signal
  18. Concurrently Observed Actions Are Represented Not as Compound Actions but as Independent Actions
  19. Plant diversity effects on aboveground and belowground N pools in temperate grassland ecosystems
  20. Das internationale Kartell- und Fusionskontrollverfahrensrecht zwischen Kooperation und Konvergenz
  21. Vertrag über die Europäische Union (EUV) : Artikel 25 [Politisches und Sicherheitspolitisches Komitee]
  22. UK should prioritise NATO, Euro-Atlantic cooperation and resilience in post-Brexit European security
  23. An analytical approach to evaluating bivariate functions of fuzzy numbers with one local extremum
  24. Orthodox Core–Heterodox Periphery? Contrasting Citation Networks of Economics Departments in Vienna
  25. Modellierung und simulation des heterogenen materialverhaltens in thermisch gespritzten schichten
  26. Drivers of intraspecific trait variation of grass and forb species in German meadows and pastures
  27. Fördert die Ganztagsschule die Entwicklung sozialer und emotionaler Kompetenzen bei Jugendlichen?