This paper improves our understanding of failed implementation in the European Union by studying a case whereby a member state has transposed legislation but failed to comply with it. Drawing on existing literature on transposition deficits, this paper creates a framework for interrogating implementation failures in the EU's multi-level governance system. It is applied to a Hungarian case study to explain why parts of the country continue to deliver drinking water that exceeds the Drinking Water Directive's limits for arsenic, creating a public health risk. The failure to comply is shown to be a series of linked implementation deficits at every level of the governance system. Processes of horizontal and vertical disintegration are clearly demonstrated. The conclusions make policy recommendations with wider relevance to other cases of failed implementation. Recommendations are also made for research that uses our framework as a starting point to understand the drivers behind individual deficits.