This paper presents the results of a study on the use of interactive online tasks to
assess students’ conceptual understanding of abstract binary operations in a firstyear linear algebra course. The assessment consists of recognition, identification and production tasks and uses verbal, graphic and symbolic representations of binary operations in numerous point set contexts. The aim of the study is to directly assess the students’ understanding of binary operations and – more indirectly – to identify different profiles for the students’ procedural and conceptual knowledge levels. A latent class analysis revealed different levels in students’ conceptual understanding. Implications will be drawn for teaching abstract binary operations – and other similar concepts. Finally, some suggestions about conceptual qualifications for mathematics teacher education will be discussed.