Case study meta-analysis in the social sciences. Insights on data quality and reliability from a large-N case survey

Research output: Journal contributionsJournal articlesResearchpeer-review

Authors

Meta-analytical methods face particular challenges in research fields such as social and political research, where studies often rest primarily on qualitative and case study research. In such contexts, where research findings are less standardized and amenable to structured synthesis, the case survey method has been proposed as a means of data generation and analysis. The method offers a meta-analytical tool to synthesize larger numbers of qualitative case studies, yielding data amenable to large-N analysis. However, resulting data is prone to specific threats to validity, including biases due to publication type, rater behaviour, and variable characteristics, which researchers need to be aware of. While these biases are well known in theory, and typically explored for primary research, their prevalence in case survey meta-analyses remains relatively unexplored. We draw on a case survey of 305 published qualitative case studies of public environmental decision-making, and systematically analyze these biases in the resultant data. Our findings indicate that case surveys can deliver high-quality and reliable results. However, we also find that these biases do indeed occur, albeit to a small degree or under specific conditions of complexity. We identify a number of design choices to mitigate biases that may threaten validity in case survey meta-analysis. Our findings are of importance to those using the case survey method – and to those who might apply insights derived by this method to inform policy and practice.
Original languageEnglish
JournalResearch Synthesis Methods
Volume13
Issue number1
Pages (from-to)12-27
Number of pages16
ISSN1759-2879
DOIs
Publication statusPublished - 01.01.2022

Bibliographical note

Publisher Copyright:
© 2021 The Authors. Research Synthesis Methods published by John Wiley & Sons Ltd.

    Research areas

  • Environmental Governance - case survey method, evidence-based governance, inter-rater reliability, meta-analysis, publication bias

Documents

DOI

Recently viewed

Activities

  1. International Conference on Advances in Social Networks Analysis and Mining - 2010
  2. 12th Interpretative Policy Analysis Conference - IPA 2017
  3. Digitalisierung der Hochschule: Studierendenzentrierung oder Exzellenzstrategie
  4. Towards a New Aesthetic Paradigm
  5. Journal of Molecular Catalysis A (Zeitschrift)
  6. Fabricating the Digital Citizen
  7. RHYTHMS OF ATTUNEMENT
  8. 6th International Conference on Soft Computing
  9. (University) support programmes and sustainable regional development: Why, how, and with what impact?
  10. Power and Potential of Artistic and Cultural Organizations in a Network of Sustainable Urban Development
  11. ICSS Group (Externe Organisation)
  12. Identifying Global Challenges for Future Tourism and Tourism Management
  13. EFFECT OF DIFFERENT RESIDUAL STRESS ENGINEERING TECHNIQUES ON THE RESIDUAL STRESS AND FATIGUE CRACK GROWTH BEHAVIOUR IN AA7075
  14. How harmonious and obsessive passion lead to entrepreneurial success: Unfolding the underlying process on a state level.
  15. In­ter­fe­ren­ces | Events
  16. Journal of Chromatographic Science (Zeitschrift)
  17. The effect of professional development on teachers’ PCK, on beliefs and on the quality of teaching
  18. Development Entrepreneurship and Personal Initiatives: Long term massive randomized experiments on personal inititiative training for entrepreneurs to reduce poverty in developing countries
  19. - Keep it simple make it fast
  20. A Soft Piezo Mechanical Hydraulic Actuator with its Liquid Stiffness Identification and its Control
  21. IWH Workshop on Labour Mobility and Inequality
  22. Positiver Aktionismus
  23. Integration of Internal Audit into Corporate Governance Systems. Empirical Evidence from the European Union
  24. The Ethics of Memory - 2009
  25. Barnboken (Zeitschrift)