Is there an excess of significant findings in published studies of psychotherapy for depression?

Research output: Journal contributionsJournal articlesResearchpeer-review

Authors

  • J. Flint
  • Pim Cuijpers
  • J. Horder
  • Sander L. Koole
  • M. R. Munafò
BACKGROUND: Many studies have examined the efficacy of psychotherapy for major depressive disorder (MDD) but publication bias against null results may exist in this literature. However, to date, the presence of an excess of significant findings in this literature has not been explicitly tested.

METHOD: We used a database of 1344 articles on the psychological treatment of depression, identified through systematic search in PubMed, PsycINFO, EMBASE and the Cochrane database of randomized trials. From these we identified 149 studies eligible for inclusion that provided 212 comparisons. We tested for an excess of significant findings using the method developed by Ioannidis and Trikalinos (2007), and compared the distribution of p values in this literature with the distribution in the antidepressant literature, where publication bias is known to be operating.

RESULTS: The average statistical power to detect the effect size indicated by the meta-analysis was 49%. A total of 123 comparisons (58%) reported a statistically significant difference between treatment and control groups, but on the basis of the average power observed, we would only have expected 104 (i.e. 49%) to do so. There was therefore evidence of an excess of significance in this literature (p = 0.010). Similar results were obtained when these analyses were restricted to studies including a cognitive behavioural therapy (CBT) arm. Finally, the distribution of p values for psychotherapy studies resembled that for published antidepressant studies, where publication bias against null results has already been established.

CONCLUSIONS: The small average size of individual psychotherapy studies is only sufficient to detect large effects. Our results indicate an excess of significant findings relative to what would be expected, given the average statistical power of studies of psychotherapy for major depression.
Original languageEnglish
JournalPsychological Medicine
Volume45
Issue number2
Pages (from-to)439-446
Number of pages8
ISSN0033-2917
DOIs
Publication statusPublished - 12.01.2015

    Research areas

  • Psychology
  • Health sciences
  • Cognitive behavioural therapy, depression, excess of significance, meta-Analysis, psychotherapy, publication bias

Recently viewed

Publications

  1. Organic farming affects the biological control of hemipteran pests and yields in spring barley independent of landscape complexity
  2. Assessing Trust by Disclosure in Online Social Networks
  3. Product diversification and stability of employment and sales
  4. Dynamische Modellierung der Sorption von Substanzen in einem hydrologischen Einzugsgebietsmodell anhand des Beispiels Phosphor
  5. Macroeconomic shocks and banks’ foreign assets
  6. Improving compliance with COVID-19 guidance
  7. Predicting expatriate job performance
  8. Alcohol intake can reduce gambling behavior
  9. Statistical Learning and Inference Is Impaired in the Nonclinical Continuum of Psychosis
  10. Schulbezogenes mathematisches Vorwissen von Bewerber*innen auf ein Mathematik-Lehramtsstudium
  11. Plastics in our ocean as transdisciplinary challenge
  12. Investigation On The Influence Of Remanufacturing On Production Planning And Control – A Systematic Literature Review
  13. Stock price reactions to climate science information from the Intergovernmental Panel on Climate Change
  14. Effects of elevated atmospheric CO2 concentrations on barley, sugar beet and wheat in a rotation
  15. Toward “hardened” accountability?
  16. New evidence for vegetation development and timing of Upper Middle Pleistocene interglacials in Northern Germany and tentative correlations
  17. Intermetallic phase characteristics in the Mg–Nd–Zn system
  18. Does Internet-based guided self-help for depression cause harm?
  19. Print exposure across the reading life span
  20. Reaching for the (Product) Stars
  21. Influence of cerium on stress corrosion cracking in AZ91D
  22. Elution of Monomers from Two Conventional Dental Composite Materials
  23. Effects of introspective vs. extraspective instruction in scaling of hedonic properties of flavouring ingredients by Chinese and German subjects
  24. Simulation of the quench sensitivity of the aluminum alloy 6082
  25. Generalising IRT to Discriminate Between Examinees
  26. Systematic study of the effect of non-uniform seal stiffness on the contact stress in flat-faced soft-seated spring operated pressure relief valves
  27. Accurate welding line prediction in extrusion processes
  28. Photochemistry of 2,2', 4,4', %,5'-hexabde (BDE-153) in THF and adsorbed on SiO2
  29. Cues from Facial Expressions for Emotional Interfaces
  30. Manufacture of profiles with variable cross-sections, curvatures and twistings by influencing the material flow in hot extrusion
  31. Effects of habitat heterogeneity on bird communities in forests of northeastern Germany
  32. The multipole resonance probe
  33. German multiple-product, multiple-destination exporters: Bernard-Redding-Schott under test
  34. Forestry contributed to warming of forest ecosystems in northern Germany during the extreme summers of 2018 and 2019
  35. In Situ Synchrotron Radiation Study of the Tension–Compression Asymmetry in an Extruded Mg–2Y–1Zn–1Mn Alloy
  36. Entrepreneurial Traits and Strategy in the Performance of Owner-manager Led Firms
  37. Beyond Personalization and Anonymity:
  38. The effect of hunting regimes on tree regeneration in lowland beech (Fagus sylvatica L.) forests
  39. „Willst du wirklich ,lehren‘ oder ,Lehrer werden‘?“