Metrics for Experimentation Programs: Categories, Benefits and Challenges
Research output: Contributions to collected editions/works › Article in conference proceedings › Research › peer-review
Authors
Experimentation programs are vital for enabling data-driven decision-making within product development. However, evaluating their overarching success remains a significant challenge. Current metrics, such as conversion rates, primarily focus on individual experiments, leaving a gap in assessing broader program efficiency and impact. This paper addresses this gap by presenting a structured overview and analysis of 18 program-level metrics, categorized into six domains: Volume, Outcome-Based, Quality, Engagement, Process Efficiency and Strategic Alignment. Metrics such as experimentation throughput, time-to-decision and experimentation coverage are examined for their implications on operational efficiency, cultural adoption, and strategic alignment. Based on interviews with 48 experimentation practitioners, this work provides a description of these metrics and discusses their benefits and challenges. The results offer actionable insights for advancing experimentation practices and aligning them with organizational goals.
Original language | English |
---|---|
Title of host publication | Agile Processes in Software Engineering and Extreme Programming : 26th International Conference on Agile Software Development, XP 2025, Brugg-Windisch, Switzerland, June 2–5, 2025, Proceedings |
Editors | Sibylle Peter, Martin Kropp, Ademar Aguiar, Craig Anslow, Maria Ilaria Lunesu, Andrea Pinna |
Number of pages | 15 |
Place of Publication | Cham |
Publisher | Springer Nature |
Publication date | 2025 |
Pages | 210-225 |
ISBN (print) | 978-3-031-94543-4 |
ISBN (electronic) | 978-3-031-94544-1 |
DOIs | |
Publication status | Published - 2025 |
Event | 26th International Conference on Agile Software Development - XP 2025 - Brugg-Windisch, Switzerland Duration: 02.06.2025 → 05.06.2025 |
- Business informatics - Continuous Experimentation, Experimentation Metrics, Program-Level Evaluation, A/B Testing