Test designs using tasks that mirror actual classroom demands are understood to predict teaching performance rather closely. Based on a methodological triangulation of empirical data from a video-based competence test, however, this article deals with a potential validity issue regarding the measurement of situation-specific skills of pre- and in-service teachers of all subjects as indicators of their competence in linguistically responsive teaching. The performance-oriented test design does not use closed items, but asks for maximally open oral responses: a verbalized perception is followed by a verbalized immediate (re-)action. The responses of n = 124 pre- and in-service teachers point towards a dilemma concerning the demand of verbalizing perceptions: The results show that the former seems to be more difficult than verbalizing immediate reactions, while the relation between both skills emerges as complicated. The nature of the dilemma is described both theoretically and empirically and thereafter put into a broader context, all the while using results from research on teachers’ expertise. Finally, consequences for measuring competences in video-based test designs are discussed.