Insights into the accuracy of social scientists’ forecasts of societal change

Research output: Journal contributionsJournal articlesResearchpeer-review

Authors

  • Igor Grossmann
  • Amanda Rotella
  • Cendri A. Hutcherson
  • Konstantyn Sharpinskyi
  • Michael E.W. Varnum
  • Sebastian Achter
  • Mandeep K. Dhami
  • Xinqi Evie Guo
  • Mane Kara-Yakoubian
  • David R. Mandel
  • Louis Raes
  • Louis Tay
  • Aymeric Vie
  • Lisa Wagner
  • Matus Adamkovic
  • Arash Arami
  • Patrícia Arriaga
  • Kasun Bandara
  • Gabriel Baník
  • František Bartoš
  • Ernest Baskin
  • Christoph Bergmeir
  • Michał Białek
  • Caroline K. Børsting
  • Dillon T. Browne
  • Eugene M. Caruso
  • Rong Chen
  • Bin Tzong Chie
  • William J. Chopik
  • Robert N. Collins
  • Chin Wen Cong
  • Lucian G. Conway
  • Matthew Davis
  • Martin V. Day
  • Nathan A. Dhaliwal
  • Justin D. Durham
  • Martyna Dziekan
  • Christian T. Elbaek
  • Eric Shuman
  • Marharyta Fabrykant
  • Mustafa Firat
  • Geoffrey T. Fong
  • Jeremy A. Frimer
  • Jonathan M. Gallegos
  • Simon B. Goldberg
  • Anton Gollwitzer
  • Julia Goyal
  • Lorenz Graf-Vlachy

How well can social scientists predict societal change, and what processes underlie their predictions? To answer these questions, we ran two forecasting tournaments testing the accuracy of predictions of societal change in domains commonly studied in the social sciences: ideological preferences, political polarization, life satisfaction, sentiment on social media, and gender–career and racial bias. After we provided them with historical trend data on the relevant domain, social scientists submitted pre-registered monthly forecasts for a year (Tournament 1; N = 86 teams and 359 forecasts), with an opportunity to update forecasts on the basis of new data six months later (Tournament 2; N = 120 teams and 546 forecasts). Benchmarking forecasting accuracy revealed that social scientists’ forecasts were on average no more accurate than those of simple statistical models (historical means, random walks or linear regressions) or the aggregate forecasts of a sample from the general public (N = 802). However, scientists were more accurate if they had scientific expertise in a prediction domain, were interdisciplinary, used simpler models and based predictions on prior data.

Original languageEnglish
JournalNature Human Behaviour
Volume7
Issue number4
Pages (from-to)484-501
Number of pages18
DOIs
Publication statusPublished - 04.2023
Externally publishedYes

Bibliographical note

Publisher Copyright:
© 2023, The Author(s), under exclusive licence to Springer Nature Limited.

Recently viewed

Researchers

  1. Lukas Kuhn

Publications

  1. Reality-based tasks for competency-based education
  2. Risk management with management control systems
  3. Ecologies of Recovery: Mónica de Miranda’s As If the World Had No West
  4. Is Calluna vulgaris a suitable bio-monitor of management-mediated nutrient pools in heathland ecosystems?
  5. Über das Essen
  6. Studying embodied encounters
  7. Lizard distribution patterns in the Tumut Fragmentation "Natural Experiment" in south-eastern Australia
  8. Study harder? the relationship of achievement goals to attitudes and self-reported use of desirable difficulties in self-regulated learning
  9. A Robust Decoupling Estimator to Indentify Electrical Parameters for Three-Phase Permanent Magnet Synchronous Motors
  10. A web- And mobile-based intervention for comorbid, recurrent depression in patients with chronic back pain on sick leave (get.back)
  11. Broad values as the basis for understanding deliberation about protected area management
  12. Special Issue: Habitual Action, Automaticity, and Control
  13. Pragmatic Competence in EIL
  14. An-arche and Indifference
  15. Self-regulated learning and self assessment in online mathematics bridging courses
  16. Adaptation strategies for reducing vulnerability to future environmental change
  17. Shared mobility business models
  18. Student Feedback as a Source for Reflection in Practical Phases of Teacher Education
  19. From Adaptive to Specific
  20. The total Archive On the Function of Not-Knowing in digital Culture
  21. HyperUCB
  22. Risky Business
  23. Selecting methods for ecosystem service assessment
  24. Klassenrat
  25. Spillover of functionally important organisms between managed and natural habitats