Is there an excess of significant findings in published studies of psychotherapy for depression?

Research output: Journal contributionsJournal articlesResearchpeer-review

Authors

  • J. Flint
  • Pim Cuijpers
  • J. Horder
  • Sander L. Koole
  • M. R. Munafò
BACKGROUND: Many studies have examined the efficacy of psychotherapy for major depressive disorder (MDD) but publication bias against null results may exist in this literature. However, to date, the presence of an excess of significant findings in this literature has not been explicitly tested.

METHOD: We used a database of 1344 articles on the psychological treatment of depression, identified through systematic search in PubMed, PsycINFO, EMBASE and the Cochrane database of randomized trials. From these we identified 149 studies eligible for inclusion that provided 212 comparisons. We tested for an excess of significant findings using the method developed by Ioannidis and Trikalinos (2007), and compared the distribution of p values in this literature with the distribution in the antidepressant literature, where publication bias is known to be operating.

RESULTS: The average statistical power to detect the effect size indicated by the meta-analysis was 49%. A total of 123 comparisons (58%) reported a statistically significant difference between treatment and control groups, but on the basis of the average power observed, we would only have expected 104 (i.e. 49%) to do so. There was therefore evidence of an excess of significance in this literature (p = 0.010). Similar results were obtained when these analyses were restricted to studies including a cognitive behavioural therapy (CBT) arm. Finally, the distribution of p values for psychotherapy studies resembled that for published antidepressant studies, where publication bias against null results has already been established.

CONCLUSIONS: The small average size of individual psychotherapy studies is only sufficient to detect large effects. Our results indicate an excess of significant findings relative to what would be expected, given the average statistical power of studies of psychotherapy for major depression.
Original languageEnglish
JournalPsychological Medicine
Volume45
Issue number2
Pages (from-to)439-446
Number of pages8
ISSN0033-2917
DOIs
Publication statusPublished - 12.01.2015

    Research areas

  • Psychology
  • Health sciences
  • Cognitive behavioural therapy, depression, excess of significance, meta-Analysis, psychotherapy, publication bias

Recently viewed

Researchers

  1. Tanja Neubüser

Projects

  1. Summer School

Publications

  1. Werbung
  2. READY! Un programma per stimolare la prontezza decisionale
  3. Evaluation and sustaining factors of machidukuri groups organized in relation with the 'hope plan'
  4. Addressing climate change with behavioral science: A global intervention tournament in 63 countries
  5. Introduction: The Political Project of Corbynism
  6. The tip of the iceberg: laptop music and the information-technological transformation of music
  7. Fachunterricht sprachsensibel gestalten mit Scaffolding
  8. Academia's obsession with quantity
  9. Regensburger Schauspiele
  10. Gender and (Un)Sustainability—Can Communication Solve a Conflict of Norms?
  11. Mitarbeiterzentriertes Qualifizierungsmanagement
  12. Die Universität im Wettbewerb
  13. A construction of singular overlapping asymmetric self-similar measures
  14. What Makes a Person a Potential Tourist and a Region a Potential Tourism Destination?
  15. Determination of sulfur and selected trace elements in metallothionein-like proteins using capillary electrophoresis hyphenated to inductively coupled plasma mass spectrometry with an octopole reaction cell
  16. Erfolgreich Führen in der Pflege
  17. A novel telecoupling framework to assess social relations across spatial scales for ecosystem services research
  18. Lektüreeindrücke zu Anselm Haverkamp
  19. Farewell to the party model?
  20. e-learning für das Fach Statistik
  21. Kontextualisierungshinweise
  22. Opportunities and limits to ecosystem services governance in developing countries and indigenous territories: The case of water supply in Southern Chile
  23. Self-Regulation, Language Skills, and Emotion Knowledge in Young Children From Northern Germany
  24. Entrepreneurship and the "theory of planned behavior"
  25. Introduction: Interiors and interiority
  26. Krieg in der Zivilgesellschaft
  27. Law, democracy, and the state of emergency.
  28. Transformative research for sustainability
  29. Coming late for dinner
  30. Property, personality rights and data protection with regard to biobanks
  31. Ordo-Responsibility
  32. How to make universal, voluntary testing for COVID-19 work? A behavioural economics perspective
  33. Nachhaltigkeit - zentrales Element des Lüneburger Studienprogramms
  34. Assessment for Transformation – Higher Education Thrives in Redefining Quality Systems
  35. Enhancing firm performance and innovativeness through error management culture
  36. 3-Lithio-1-triisopropylsilyl-1-propyne

Press / Media

  1. App gegen Wegwerf-Mode