TY - CHAP

T1 - In-House Experimentation Platforms Motivations, Implementation Characteristics and Challenges

T2 - 26th International Conference on Product-Focused Software Process Improvement

AU - Stotz, Nils

AU - Drews, Paul

N1 - Conference code: 26

PY - 2026

Y1 - 2026

N2 - In-house experimentation platforms are increasingly used to support continuous, data-driven product development. While prior research outlines general infrastructure requirements, it offers limited insight into why companies build their own platforms, how they design them, and what challenges emerge. This study examines publicly available industry reports and engineering blogs to identify recurring motivations, implementation patterns, and organizational challenges. Companies pursue in-house solutions to gain context-specific functionality, ensure compliance, and embed experimentation into workflows. These efforts lead to modular architectures, custom metrics pipelines, and self-service tooling—but also introduce challenges such as scalability limits, knowledge silos, and cultural resistance. A process model illustrates how motivations shape implementation and how challenges drive iterative refinement. The findings position platforms not as neutral tools but as evolving socio-technical systems embedded in organizational context. The study distinguishes in-house platforms from off-the-shelf solutions and offers practical insights into build-vs-buy decisions, design trade-offs, and long-term experimentation strategy.

AB - In-house experimentation platforms are increasingly used to support continuous, data-driven product development. While prior research outlines general infrastructure requirements, it offers limited insight into why companies build their own platforms, how they design them, and what challenges emerge. This study examines publicly available industry reports and engineering blogs to identify recurring motivations, implementation patterns, and organizational challenges. Companies pursue in-house solutions to gain context-specific functionality, ensure compliance, and embed experimentation into workflows. These efforts lead to modular architectures, custom metrics pipelines, and self-service tooling—but also introduce challenges such as scalability limits, knowledge silos, and cultural resistance. A process model illustrates how motivations shape implementation and how challenges drive iterative refinement. The findings position platforms not as neutral tools but as evolving socio-technical systems embedded in organizational context. The study distinguishes in-house platforms from off-the-shelf solutions and offers practical insights into build-vs-buy decisions, design trade-offs, and long-term experimentation strategy.

KW - Business informatics

KW - A/B Testing

KW - Continuous Discovery

KW - Continuous Experimentation

KW - Experimentation Platform

KW - Hypothesis Testing

UR - http://www.scopus.com/inward/record.url?scp=105023314358&partnerID=8YFLogxK

U2 - 10.1007/978-3-032-12089-2_3

DO - 10.1007/978-3-032-12089-2_3

M3 - Article in conference proceedings

SN - 978-3-032-12088-5

T3 - Lecture Notes in Computer Science

SP - 36

EP - 51

BT - Product-Focused Software Process Improvement

A2 - Scanniello, Giuseppe

A2 - Lenarduzzi, Valentina

A2 - Romano, Simone

A2 - Vegas, Sira

A2 - Francese, Rita

PB - Springer Nature Switzerland AG

CY - Cham

Y2 - 1 December 2025 through 3 December 2025

ER -