Case study meta-analysis in the social sciences. Insights on data quality and reliability from a large-N case survey

Research output: Journal contributionsJournal articlesResearchpeer-review

Authors

Meta-analytical methods face particular challenges in research fields such as social and political research, where studies often rest primarily on qualitative and case study research. In such contexts, where research findings are less standardized and amenable to structured synthesis, the case survey method has been proposed as a means of data generation and analysis. The method offers a meta-analytical tool to synthesize larger numbers of qualitative case studies, yielding data amenable to large-N analysis. However, resulting data is prone to specific threats to validity, including biases due to publication type, rater behaviour, and variable characteristics, which researchers need to be aware of. While these biases are well known in theory, and typically explored for primary research, their prevalence in case survey meta-analyses remains relatively unexplored. We draw on a case survey of 305 published qualitative case studies of public environmental decision-making, and systematically analyze these biases in the resultant data. Our findings indicate that case surveys can deliver high-quality and reliable results. However, we also find that these biases do indeed occur, albeit to a small degree or under specific conditions of complexity. We identify a number of design choices to mitigate biases that may threaten validity in case survey meta-analysis. Our findings are of importance to those using the case survey method – and to those who might apply insights derived by this method to inform policy and practice.
Original languageEnglish
JournalResearch Synthesis Methods
Volume13
Issue number1
Pages (from-to)12-27
Number of pages16
ISSN1759-2879
DOIs
Publication statusPublished - 01.01.2022

Bibliographical note

Publisher Copyright:
© 2021 The Authors. Research Synthesis Methods published by John Wiley & Sons Ltd.

    Research areas

  • Environmental Governance - case survey method, evidence-based governance, inter-rater reliability, meta-analysis, publication bias

Documents

DOI

Recently viewed

Activities

  1. Personal care products as source for micropollutants in Greywater-Identification, quantification and on-site treatment
  2. How to get things done with words in Irish English and English English: Pragmatics and dialectology at the cross-road
  3. „Theorie und Praxis: Globales Lernen und Bildung für eine nachhaltige Entwicklung – Positionen, Potentiale, Perspektiven“
  4. Integrating Systems Thinking Approaches in Chemical Education for Sustainable Development - Exploring the Status Quo
  5. Advantages and challenges of in silico tools and their contribution to sustainable chemistry with the focus on (Q)SAR
  6. Sustainability and internal supply chain management – A conceptual approach to intra-organisational knowledge transfer
  7. Degradation of the emerging contaminant Atenolol by K2FeVIO4: assessment of biodegradability and degradation products
  8. Pessimism and Optimism of the Future: the Future of the World(s) in the Strategies of Feminism and Digital Technologies
  9. People incorrectly correcting other people“: (Re-)Correcting comments as a means of stancetaking in digital communication
  10. Exploring priority effects in a central European grassland field experiment in order to inform restoration (Symposium)
  11. Kreativ und frei?: Welche Rahmenbedingungen brauchen Kreativwirtschaft und Wissensgesellschaft in einer digitalen Welt?
  12. Das Leuphana-College: ein in Europaeinzigartiges Hochschulmodell: Innovative hochschuldidaktische Veranstaltungsformate
  13. Deciphering Sustainable Consumption: Understanding, Motives and Heuristic Cues in the Context of Personal Care Product.
  14. Degradation of the emerging contaminant Atenolol by K2FeVIO4: assessment of biodegradability and degradation products

Publications

  1. Hydration and dehydration of salt hydrates and hydroxides for thermal energy storage – kinetics and energy release
  2. Establishment success in a forest biodiversity and ecosystem functioning experiment in subtropical China (BEF-China)
  3. Gerhard O. Forde: The Captivation of the Will. Luther vs. Erasmus on Freedom and Bondage, Grand Rapids / Cambridge 2005
  4. Menschliche Würde in der Lebensphase natürlicher Abhängigkeit - Jugendpsychiatrische und sozialpädaogische Aspekte
  5. Using heuristic worked examples to promote solving of reality‑based tasks in mathematics in lower secondary school
  6. The Shareholder Value Effect of System Overloads: An Analysis of Investor Responses to the 2003 Blackout in the US
  7. How do workers gain voice on digital work platforms? Hotspots and blind spots in research on platform worker voice
  8. Do we need livestock grazing to promote Polylepis australis tree recruitment in the Central Argentinean Mountains?
  9. Planungs- und Genehmigungsfragen bei der Finanzierung von Anlagen zur Erzeugung von Strom aus Erneuerbaren Energien
  10. Model-based estimation of pesticides and transformation products and their export pathways in a headwater catchment
  11. Reciprocal relations between emotional exhaustion and episode-specific emotional labour: An experience-sampling study
  12. Utilizing international networks for accelerating research and learning in transformational sustainability science
  13. Mathematical Modeling for Robot 3D Laser Scanning in Complete Darkness Environments to Advance Pipeline Inspection
  14. Einfluss elterlicher Mathematikkompetenz und familialer Prozesse auf den Kompetenzerwerb von Kindern in Mathematik
  15. Perfluorooctanoic acid (PFOA)-main concerns and regulatory developments in Europe from an environmental point of view
  16. Tree and mycorrhizal fungal diversity drive intraspecific and intraindividual trait variation in temperate forests
  17. Effects of anthropogenic disturbances on soil microbial communities in oak forests persist for more than 100 years
  18. Den eigenen Blick schärfen für das Potenzial und die Qualität von Aufgaben – Ergebnisse aus der aktuellen Forschung
  19. Social and ethical aspects in sustainability performance measurement and assessment. A systematic literature review
  20. What is intergovernmental about the EU’s ‘(new) intergovernmentalist’ turn? Evidence from the Eurozone and asylum crises
  21. Schulentwicklungsbegleitung auf Augenhöhe – die Besonderheiten des Beratungsansatzes der regionalen SchuMaS-Zentren
  22. Tobias Cheung: Organismen - Agenten zwischen Innen- und Außenwelt 1780-1860, (Science Studies) Bielefeld: transcript 2014
  23. Methane yield of biomass from extensive grassland is affected by compositional changes induced by order of arrival
  24. Kraft ist nicht gleich Kraft. Und wie können die Kraftfähigkeiten spielerisch in der Grundschule vermittelt werden?“