Although habitat loss is the predominant factor leading to biodiversity loss in the Anthropocene^1,2, exactly how this loss manifests—and at which scales—remains a central debate^3–6. The ‘passive sampling’ hypothesis suggests that species are lost in proportion to their abundance and distribution in the natural habitat^7,8, whereas the ‘ecosystem decay’ hypothesis suggests that ecological processes change in smaller and more-isolated habitats such that more species are lost than would have been expected simply through loss of habitat alone^9,10. Generalizable tests of these hypotheses have been limited by heterogeneous sampling designs and a narrow focus on estimates of species richness that are strongly dependent on scale. Here we analyse 123 studies of assemblage-level abundances of focal taxa taken from multiple habitat fragments of varying size to evaluate the influence of passive sampling and ecosystem decay on biodiversity loss. We found overall support for the ecosystem decay hypothesis. Across all studies, ecosystems and taxa, biodiversity estimates from smaller habitat fragments—when controlled for sampling effort—contain fewer individuals, fewer species and less-even communities than expected from a sample of larger fragments. However, the diversity loss due to ecosystem decay in some studies (for example, those in which habitat loss took place more than 100 years ago) was less than expected from the overall pattern, as a result of compositional turnover by species that were not originally present in the intact habitats. We conclude that the incorporation of non-passive effects of habitat loss on biodiversity change will improve biodiversity scenarios under future land use, and planning for habitat protection and restoration.