Primate cognition research allows us to reconstruct the evolution of human cognition. However, temporal and contextual factors that induce variation in cognitive studies with great apes are poorly understood. Here we report on a longitudinal study where we repeatedly tested a comparatively large sample of great apes (N = 40) with the same set of cognitive measures. We investigated the stability of group-level results, the reliability of individual differences, and the relation between cognitive performance and individual-level characteristics. We found results to be relatively stable on a group level. Some, but not all, tasks showed acceptable levels of reliability. Cognitive performance across tasks was not systematically related to any particular individual-level predictor. This study highlights the importance of methodological considerations – especially when studying individual differences – on the route to building a more robust science of primate cognitive evolution.