Insights into the accuracy of social scientists’ forecasts of societal change
Research output: Journal contributions › Journal articles › Research › peer-review
Standard
In: Nature Human Behaviour, Vol. 7, No. 4, 04.2023, p. 484-501.
Research output: Journal contributions › Journal articles › Research › peer-review
Harvard
APA
Vancouver
Bibtex
}
RIS
TY - JOUR
T1 - Insights into the accuracy of social scientists’ forecasts of societal change
AU - Grossmann, Igor
AU - Rotella, Amanda
AU - Hutcherson, Cendri A.
AU - Sharpinskyi, Konstantyn
AU - Varnum, Michael E.W.
AU - Achter, Sebastian
AU - Dhami, Mandeep K.
AU - Guo, Xinqi Evie
AU - Kara-Yakoubian, Mane
AU - Mandel, David R.
AU - Raes, Louis
AU - Tay, Louis
AU - Vie, Aymeric
AU - Wagner, Lisa
AU - Adamkovic, Matus
AU - Arami, Arash
AU - Arriaga, Patrícia
AU - Bandara, Kasun
AU - Baník, Gabriel
AU - Bartoš, František
AU - Baskin, Ernest
AU - Bergmeir, Christoph
AU - Białek, Michał
AU - Børsting, Caroline K.
AU - Browne, Dillon T.
AU - Caruso, Eugene M.
AU - Chen, Rong
AU - Chie, Bin Tzong
AU - Chopik, William J.
AU - Collins, Robert N.
AU - Cong, Chin Wen
AU - Conway, Lucian G.
AU - Davis, Matthew
AU - Day, Martin V.
AU - Dhaliwal, Nathan A.
AU - Durham, Justin D.
AU - Dziekan, Martyna
AU - Elbaek, Christian T.
AU - Shuman, Eric
AU - Fabrykant, Marharyta
AU - Firat, Mustafa
AU - Fong, Geoffrey T.
AU - Frimer, Jeremy A.
AU - Gallegos, Jonathan M.
AU - Goldberg, Simon B.
AU - Gollwitzer, Anton
AU - Goyal, Julia
AU - Graf-Vlachy, Lorenz
AU - Krenzler, Ruslan
AU - Sevincer, Timur
N1 - Publisher Copyright: © 2023, The Author(s), under exclusive licence to Springer Nature Limited.
PY - 2023/4
Y1 - 2023/4
N2 - How well can social scientists predict societal change, and what processes underlie their predictions? To answer these questions, we ran two forecasting tournaments testing the accuracy of predictions of societal change in domains commonly studied in the social sciences: ideological preferences, political polarization, life satisfaction, sentiment on social media, and gender–career and racial bias. After we provided them with historical trend data on the relevant domain, social scientists submitted pre-registered monthly forecasts for a year (Tournament 1; N = 86 teams and 359 forecasts), with an opportunity to update forecasts on the basis of new data six months later (Tournament 2; N = 120 teams and 546 forecasts). Benchmarking forecasting accuracy revealed that social scientists’ forecasts were on average no more accurate than those of simple statistical models (historical means, random walks or linear regressions) or the aggregate forecasts of a sample from the general public (N = 802). However, scientists were more accurate if they had scientific expertise in a prediction domain, were interdisciplinary, used simpler models and based predictions on prior data.
AB - How well can social scientists predict societal change, and what processes underlie their predictions? To answer these questions, we ran two forecasting tournaments testing the accuracy of predictions of societal change in domains commonly studied in the social sciences: ideological preferences, political polarization, life satisfaction, sentiment on social media, and gender–career and racial bias. After we provided them with historical trend data on the relevant domain, social scientists submitted pre-registered monthly forecasts for a year (Tournament 1; N = 86 teams and 359 forecasts), with an opportunity to update forecasts on the basis of new data six months later (Tournament 2; N = 120 teams and 546 forecasts). Benchmarking forecasting accuracy revealed that social scientists’ forecasts were on average no more accurate than those of simple statistical models (historical means, random walks or linear regressions) or the aggregate forecasts of a sample from the general public (N = 802). However, scientists were more accurate if they had scientific expertise in a prediction domain, were interdisciplinary, used simpler models and based predictions on prior data.
KW - Psychology
UR - http://www.scopus.com/inward/record.url?scp=85147648888&partnerID=8YFLogxK
UR - https://www.mendeley.com/catalogue/4dc17226-e243-3733-837b-65d5ed75c63d/
U2 - 10.1038/s41562-022-01517-1
DO - 10.1038/s41562-022-01517-1
M3 - Journal articles
C2 - 36759585
AN - SCOPUS:85147648888
VL - 7
SP - 484
EP - 501
JO - Nature Human Behaviour
JF - Nature Human Behaviour
SN - 2397-3374
IS - 4
ER -