Metrics for Experimentation Programs: Categories, Benefits and Challenges

Research output: Contributions to collected editions/worksArticle in conference proceedingsResearchpeer-review

Authors

Experimentation programs are vital for enabling data-driven decision-making within product development. However, evaluating their overarching success remains a significant challenge. Current metrics, such as conversion rates, primarily focus on individual experiments, leaving a gap in assessing broader program efficiency and impact. This paper addresses this gap by presenting a structured overview and analysis of 18 program-level metrics, categorized into six domains: Volume, Outcome-Based, Quality, Engagement, Process Efficiency and Strategic Alignment. Metrics such as experimentation throughput, time-to-decision and experimentation coverage are examined for their implications on operational efficiency, cultural adoption, and strategic alignment. Based on interviews with 48 experimentation practitioners, this work provides a description of these metrics and discusses their benefits and challenges. The results offer actionable insights for advancing experimentation practices and aligning them with organizational goals.
Original languageEnglish
Title of host publicationAgile Processes in Software Engineering and Extreme Programming : 26th International Conference on Agile Software Development, XP 2025, Brugg-Windisch, Switzerland, June 2–5, 2025, Proceedings
EditorsSibylle Peter, Martin Kropp, Ademar Aguiar, Craig Anslow, Maria Ilaria Lunesu, Andrea Pinna
Number of pages16
Place of PublicationCham
PublisherSpringer Nature
Publication date2025
Pages210-225
ISBN (print)978-3-031-94543-4
ISBN (electronic)978-3-031-94544-1
DOIs
Publication statusPublished - 2025
Event26th International Conference on Agile Software Development - XP 2025 - Brugg-Windisch, Switzerland
Duration: 02.06.202505.06.2025

Bibliographical note

Publisher Copyright:
© The Author(s) 2025.

Recently viewed

Publications

  1. Defining the notion of mining, extraction and collection
  2. Exploring Leverages and Pitfalls of Context Collapse in Modern Communication
  3. Combining sense of place theory with the ecosystem services concept: empirical insights and reflections from a participatory mapping study
  4. Does online-delivered Cognitive Behavioural Therapy for Insomnia improve insomnia severity in nurses working shifts? Protocol for a randomised-controlled trial
  5. Non-invariance? An Overstated Problem With Misconceived Causes
  6. Efficacy of a web-based intervention with and without guidance for employees with risky drinking
  7. The polarity field concept
  8. Guest Editors' Introduction
  9. Question Answering Mediated by Visual Clues and Knowledge Graphs
  10. Understanding Similarities and Differences of Digital Health Platforms
  11. Environmental Shareholder Value Matrix
  12. Rechtschreiben
  13. Bierbrausieb
  14. Ideological Foundations of Perceived Contract Breach Associated With Downsizing
  15. Umweltrechtsschutz in China
  16. 11. Methoden-Muster
  17. The use of force against terrorists
  18. Transparency in an Age of Digitalization and Responsibility
  19. Understanding the bright side and the dark side of telework
  20. Effect of free air carbon dioxide enrichment combined with two nitrogen levels on growth, yield and yield quality of sugar beet
  21. Caring, Helping, and For-giving
  22. Managing Research Environments
  23. Pesticide peak concentration reduction in a small vegetated treatment system controlled by chemograph shape
  24. Thermal analysis of laser additive manufacturing of aluminium alloys: Experiment and simulation
  25. Quality Education and lifelong learning for all: trying to get to grips with the iridescent, multifaced, and at the same time universal character of SDG 4
  26. Sustainable engineering education in research and practice
  27. Reversed effects of grazing on plant diversity