Because biodiversity is multidimensional and scale-dependent, it is challenging to estimate its change. However, it is unclear (1) how much scale-dependence matters for empirical studies, and (2) if it does matter, how exactly we should quantify biodiversity change. To address the first question, we analysed studies with comparisons among multiple assemblages, and found that rarefaction curves frequently crossed, implying reversals in the ranking of species richness across spatial scales. Moreover, the most frequently measured aspect of diversity – species richness – was poorly correlated with other measures of diversity. Second, we collated studies that included spatial scale in their estimates of biodiversity change in response to ecological drivers and found frequent and strong scale-dependence, including nearly 10% of studies which showed that biodiversity changes switched directions across scales. Having established the complexity of empirical biodiversity comparisons, we describe a synthesis of methods based on rarefaction curves that allow more explicit analyses of spatial and sampling effects on biodiversity comparisons. We use a case study of nutrient additions in experimental ponds to illustrate how this multi-dimensional and multi-scale perspective informs the responses of biodiversity to ecological drivers.