If tests of cognitive ability are repeatedly taken, test scores rise. Such retest effects have been observed for a long time and for a variety of tasks. This study investigates retest effects on figural matrix items in an educational context. A short term effect is assumed for the direct retest administration in the same test session, and a long term effect is assumed for a retest interval of six months. Using multilevel modeling, we analyze if the magnitude of these effects is not only influenced by individual variation, but also by the cluster structure of students grouped within classrooms. We also investigate if the use of identical versus parallel tests has an impact on the size of the retest effects. Our main results show a negligible short term retest effect, but a large long term retest effect. Using parallel tests does not contribute to understanding individual differences in retest effects. The variation in retest effects is larger between classrooms than between students. Reasoning ability, as measured with a different test, and school grades significantly influences retest effects at the individual level, but at the classroom level, only reasoning ability is a significant predictor.