Analytic reproducibility in articles receiving open data badges at the journal Psychological Science: An observational study

Publikation: Beiträge in ZeitschriftenZeitschriftenaufsätzeForschungbegutachtet

Authors

  • Tom E. Hardwicke
  • Manuel Bohn
  • Kyle MacDonald
  • Emily Hembacher
  • Michèle B. Nuijten
  • Benjamin N. Peloquin
  • Benjamin E. Demayo
  • Bria Long
  • Erica J. Yoon
  • Michael C. Frank

For any scientific report, repeating the original analyses upon the original data should yield the original outcomes. We evaluated analytic reproducibility in 25 Psychological Science articles awarded open data badges between 2014 and 2015. Initially, 16 (64%, 95% confidence interval [43,81]) articles contained at least one 'major numerical discrepancy' (>10% difference) prompting us to request input from original authors. Ultimately, target values were reproducible without author involvement for 9 (36% [20,59]) articles; reproducible with author involvement for 6 (24% [8,47]) articles; not fully reproducible with no substantive author response for 3 (12% [0,35]) articles; and not fully reproducible despite author involvement for 7 (28% [12,51]) articles. Overall, 37 major numerical discrepancies remained out of 789 checked values (5% [3,6]), but original conclusions did not appear affected. Non-reproducibility was primarily caused by unclear reporting of analytic procedures. These results highlight that open data alone is not sufficient to ensure analytic reproducibility.

OriginalspracheEnglisch
Aufsatznummer201494
ZeitschriftRoyal Society Open Science
Jahrgang8
Ausgabenummer1
Anzahl der Seiten9
ISSN2054-5703
DOIs
PublikationsstatusErschienen - 06.01.2021
Extern publiziertJa

Bibliographische Notiz

Publisher Copyright:
© 2021 The Authors.

Dokumente

DOI

Zuletzt angesehen

Publikationen

  1. Foreword to applied data science, demo, and nectar tracks
  2. Learning with summaries
  3. Is Code Law? Kritik in Zeiten algorithmischer Gouvernementalität
  4. Using Daily Stretching to Counteract Performance Decreases as a Result of Reduced Physical Activity—A Controlled Trial
  5. Polynomial Augmented Extended Kalman Filter to Estimate the State of Charge of Lithium-Ion Batteries
  6. DECODING SUSTAINABILITY IN THE HEALTHCARE SYSTEM. TEACHING STUDENTS HOW TO PROBLEMATIZE COMPLEX CONCEPTS
  7. Learning Analytics an Hochschulen
  8. Sustainability-oriented technology exploration: managerial values, ambidextrous design, and separation drift
  9. Tree mixtures mediate negative effects of introduced tree species on bird taxonomic and functional diversity
  10. Do we fail to exert self-control because we lack resources or motivation? Competing theories to explain a debated phenomenon
  11. Measurement invariance in a grid-based measure of academic self-concept
  12. Lessons learned and challenges for environmental management in Colombia
  13. How and Why Precise Anchors Distinctly Affect Anchor Recipients and Senders
  14. Interpersonal Physiological Synchrony Predicts Group Cohesion
  15. The Effectivity of Technological Innovation on Mitigating the Costs of Climate Change Policies
  16. Inclusive conservation and the Post-2020 Global Biodiversity Framework
  17. Distal and proximal predictors of snacking at work
  18. The 'need for speed'
  19. The Application of Extended Producer Responsibility in Germany
  20. Lernen in Netzwerken
  21. Recognising the role of local and Indigenous communities in managing natural resources for the greater public benefit
  22. Facing complex crime
  23. Mapping and analysing historical indicators of ecosystem services in Germany
  24. Inequality in the Transition from Primary to Secondary School
  25. Effects of gadolinium and neodymium addition on young’s modulus of magnesium-based binary alloys
  26. The Use of Media in Intercultural Dialogue "dialogo_dialog"!
  27. Mental models and attentional processes in car driving
  28. Realist Inquiry
  29. Establishing the next generation at work
  30. A hysteresis hybrid extended kalman filter as an observer for sensorless valve control in camless internal combustion engines
  31. Comfort and Adaptive Cruise Control in Highly Automated Vehicles
  32. I'd prefer not to
  33. Migration in der Adoleszenz
  34. From Point of Sale to Point of Need
  35. Insights into an Action-Oriented Training Program to Promote Sustainable Entrepreneurship
  36. Sustainability in Business: Integrated Management of Value Creation and Disvalue Mitigation
  37. After Occupy
  38. DSM-IV und DSM-5
  39. A Dual Kalman Filter to Identify Parameters of a Permanent Magnet Synchronous Motor
  40. ‚Descriptio ancilla narrationis‘?