On the relational structure of two tests measuring general pedagogical knowledge
Activity: Talk or presentation › Guest lectures › Research
Andreas Seifert - Oral presentation
Teacher education at university level, especially in the area of general pedagogy, is deemed to be highly arbitrary. In addition, there is little evidence for the effectiveness of degree programs in this field (Oelkers & Tenorth, 1993; Terhart, 2000; Tenorth, 2006). One reason for this allegation is the fact that the development of standardized methods to measure general pedagogical knowledge is a new approach in the field of empirical research on teacher education (Schaefers, 2002; Terhart, 2002; Baumert & Kunter, 2006; Blömeke, Kaiser & Lehmann, 2008; 2010a, b; Klieme & Leutner, 2006). Furthermore, there is a lack of longitudinal research which delivers insights on how future teachers acquire general pedagogical knowledge. Against this background, the aim of the study “LEK” (Längsschnittliche Erhebung pädagogischer Kompetenzen von Lehramtsstudierenden) is to investigate in detail and to model longitudinally the development in general pedagogical knowledge of future teachers by using two different test conceptions measuring general pedagogical knowledge.
Research questions
How do the constructs in both tests relate to each other?
Do the tests measure different aspects of general pedagogical knowledge?
Are there considerable intersections between the different aspects of general pedagogical knowledge?
Is there an increase in general pedagogical knowledge over time spent at university?
Do certain aspects of general pedagogical knowledge show a larger increase than others?
Methodology
The LEK-Study contains two tests for general pedagogical knowledge which were applied at the beginning and in the middle of the course of studies at two universities (Paderborn, Passau) in two German federal states (n = 525). The first test focuses on knowledge of instruction and assessment (König et al., 2011); the second test measures a broader concept which includes knowledge of education, school development and society in addition (Seifert et al., 2009, Seifert & Schaper, 2010). The first test contains four different areas of content (structuring instruction – classroom management/motivation – dealing with heterogeneity of students – assessment) and three areas of cognitive requirement (recalling – understanding/analyzing – generating). The second test contains three different areas of content (educational background – general didactics – school development and society) and three areas of cognitive requirement (reproducing/understanding – reflecting/applying – judging/evaluating/deciding). To shed light on the above-mentioned questions, several one- and multidimensional partial-credit-models were calculated from the LEK-data using the statistical software ConQuest (Wu, Adams & Wilson, 2002). Then the exported WLE-Scores (Weighted Likelihood Estimates), each reflecting a construct, were correlated. Additionally, mean differences in the WLE-Scores over time were calculated and tested for each construct.
Findings
The description of the relational structure in the above-mentioned constructs gives insights into the construct validity of both tests. The empirical findings widely support theoretical assumptions which were attained from qualitative comparisons of both tests. Furthermore, it can be observed that both instruments measured a significant increase in knowledge thoroughly relevant to all areas of pedagogical knowledge at both universities. This means, on the one hand, that degree programs dealing with general pedagogy are not ineffective. This is the first empirical evidence for the effectiveness of such degree programs, as far as the authors know. On the other hand, the findings can be regarded as a proof of the content validity of both instruments. Taking a deeper look, the amount of knowledge acquisition varies considerably across the different constructs. In the area of content the largest effects can be recognized in the field of dealing with heterogeneity of students in the first test and in the field of general didactics in the second test. This finding is very plausible because these topics are highlighted in the study and examination regulations in the first two years of the degree program. In the area of cognitive requirement the largest effects can be recognized in the fields of recalling and understanding/analyzing in the first test and in the field of reproducing/understanding in the second test. This finding may be a result of the fact that in the beginning of the degree program students predominantly have to attend to lectures rather than seminars to get a general overview of several topics. This setting is a probable cause for students gaining rather declarative than action-oriented and situated knowledge.
Discussion
The development of the two tests measuring general pedagogical knowledge, which can both display the acquisition of knowledge of future teachers in a valid way, enables education researchers and education politicians to assure and develop the quality of degree programs. Linking the outcome of the assessment of future teachers to attributes of degree programs can help to improve didactic structures at the universities based on empirical evidence. In a long-term but also sustainable perspective this may help to prepare future teachers better to do their job well and work professionally.
References:
Baumert, J. & Kunter, M. (2006). Stichwort: Professionelle Kompetenz von Lehrkräften. Zeitschrift für Erziehungswissenschaft, 9, 469-520.
Blömeke, S., Kaiser, G. & Lehmann, R. (Eds.) (2008). Professionelle Kompetenz angehender Lehrerinnen und Lehrer. Wissen, Überzeugungen und Lerngelegenheiten deutscher Mathematikstudierender und -referendare - Erste Ergebnisse zur Wirksamkeit der Lehrerausbildung. Münster: Waxmann.
Blömeke, S., Kaiser, G., & Lehmann, R. (Hrsg.) (2010a). TEDS-M 2008 – Professionelle Kompetenz und Lerngelegenheiten angehender Primarstufenlehrkräfte im internationalen Vergleich. Münster: Waxmann.
Blömeke, S., Kaiser, G., & Lehmann, R. (Hrsg.) (2010b). TEDS-M 2008 – Professionelle Kompetenz und Lerngelegenheiten angehender Mathematiklehrkräfte im internationalen Vergleich. Münster: Waxmann.
Klieme, E. & Leutner, D. (2006). Kompetenzmodelle zur Erfassung individueller Lernergebnisse und zur Bilanzierung von Bildungsprozessen. Zeitschrift für Pädagogik, 6, 876-903.König, J. & Blömeke, S. (2009c). Disziplin- oder Berufsorientierung? Zur Struktur des pädagogischen Wissens angehender Lehrkräfte. Lehrerbildung auf dem Prüfstand, 2 (1), 126-147.
König, J. , Blömeke, S., Paine, L., Schmidt, B. & Feng-Jui Hsieh (2011). General Pedagogical Knowledge of Future Middle School Teachers. On the Complex Ecology of Teacher Education in the United States, Germany, and Taiwan. Journal of Teacher Education, 62 (2), 188-201.
Oelkers, J. & Tenorth, H.-E. (Hrsg.) (1993). Pädagogisches Wissen. Weinheim: Beltz.
Schaefers, C. (2002). Forschung zur Lehrerausbildung in Deutschland - eine bilanzierende Übersicht der neueren empirischen Studien. Schweizerische Zeitschrift für Bildungswissenschaften, 24, 65-88.
Seifert, A. & Schaper, N. (2010). Überprüfung eines Kompetenzmodells und Messinstruments zur Strukturierung allgemeiner pädagogischer Kompetenz in der universitären Lehrerbildung. Lehrerbildung auf dem Prüfstand, 3 (2), 179-198.
Seifert, A., Hilligus, A. H., & Schaper, N. (2009). Entwicklung und psychometrische Überprüfung eines Messinstrumentes zur Erfassung pädagogischer Kompetenzen in der universitären Lehrerbildung. Lehrerbildung auf dem Prüfstand, 2 (1), 82-103.
Tenorth, H.-E. (2006). Professionalität im Lehrerberuf. Ratlosigkeit der Theorie, gelingende Praxis. Zeitschrift für Erziehungswissenschaft, 9, 580-597.
Terhart, E. (Ed.) (2000). Perspektiven der Lehrerbildung in Deutschland. Abschlussbericht der von der Kultusministerkonferenz eingesetzten Kommission. Weinheim: Beltz.
Terhart, E. (2002). Standards für die Lehrerbildung. Eine Expertise für die Kultusministerkonferenz. Münster: Universität Münster.
Wu, M. L., Adams, R. J., & Wilson, M. R. (1997). ConQuest: Multi-Aspect Test Software [Computer program]. Camberwell, Vic.: Australian Council for Educational Research.
Three questions to be discussed with the audience
1. How can the instruments of general pedagogical knowledge be criterion-validated (e.g. rating the quality of teacher behavior in videos of lessons, or measuring stress hormone level after class)?
2. Are the operationalizations of general pedagogical knowledge valid in other (European) nations?
3. Is general pedagogical knowledge more or less important than content knowledge to be a successful and professional teacher?
Research questions
How do the constructs in both tests relate to each other?
Do the tests measure different aspects of general pedagogical knowledge?
Are there considerable intersections between the different aspects of general pedagogical knowledge?
Is there an increase in general pedagogical knowledge over time spent at university?
Do certain aspects of general pedagogical knowledge show a larger increase than others?
Methodology
The LEK-Study contains two tests for general pedagogical knowledge which were applied at the beginning and in the middle of the course of studies at two universities (Paderborn, Passau) in two German federal states (n = 525). The first test focuses on knowledge of instruction and assessment (König et al., 2011); the second test measures a broader concept which includes knowledge of education, school development and society in addition (Seifert et al., 2009, Seifert & Schaper, 2010). The first test contains four different areas of content (structuring instruction – classroom management/motivation – dealing with heterogeneity of students – assessment) and three areas of cognitive requirement (recalling – understanding/analyzing – generating). The second test contains three different areas of content (educational background – general didactics – school development and society) and three areas of cognitive requirement (reproducing/understanding – reflecting/applying – judging/evaluating/deciding). To shed light on the above-mentioned questions, several one- and multidimensional partial-credit-models were calculated from the LEK-data using the statistical software ConQuest (Wu, Adams & Wilson, 2002). Then the exported WLE-Scores (Weighted Likelihood Estimates), each reflecting a construct, were correlated. Additionally, mean differences in the WLE-Scores over time were calculated and tested for each construct.
Findings
The description of the relational structure in the above-mentioned constructs gives insights into the construct validity of both tests. The empirical findings widely support theoretical assumptions which were attained from qualitative comparisons of both tests. Furthermore, it can be observed that both instruments measured a significant increase in knowledge thoroughly relevant to all areas of pedagogical knowledge at both universities. This means, on the one hand, that degree programs dealing with general pedagogy are not ineffective. This is the first empirical evidence for the effectiveness of such degree programs, as far as the authors know. On the other hand, the findings can be regarded as a proof of the content validity of both instruments. Taking a deeper look, the amount of knowledge acquisition varies considerably across the different constructs. In the area of content the largest effects can be recognized in the field of dealing with heterogeneity of students in the first test and in the field of general didactics in the second test. This finding is very plausible because these topics are highlighted in the study and examination regulations in the first two years of the degree program. In the area of cognitive requirement the largest effects can be recognized in the fields of recalling and understanding/analyzing in the first test and in the field of reproducing/understanding in the second test. This finding may be a result of the fact that in the beginning of the degree program students predominantly have to attend to lectures rather than seminars to get a general overview of several topics. This setting is a probable cause for students gaining rather declarative than action-oriented and situated knowledge.
Discussion
The development of the two tests measuring general pedagogical knowledge, which can both display the acquisition of knowledge of future teachers in a valid way, enables education researchers and education politicians to assure and develop the quality of degree programs. Linking the outcome of the assessment of future teachers to attributes of degree programs can help to improve didactic structures at the universities based on empirical evidence. In a long-term but also sustainable perspective this may help to prepare future teachers better to do their job well and work professionally.
References:
Baumert, J. & Kunter, M. (2006). Stichwort: Professionelle Kompetenz von Lehrkräften. Zeitschrift für Erziehungswissenschaft, 9, 469-520.
Blömeke, S., Kaiser, G. & Lehmann, R. (Eds.) (2008). Professionelle Kompetenz angehender Lehrerinnen und Lehrer. Wissen, Überzeugungen und Lerngelegenheiten deutscher Mathematikstudierender und -referendare - Erste Ergebnisse zur Wirksamkeit der Lehrerausbildung. Münster: Waxmann.
Blömeke, S., Kaiser, G., & Lehmann, R. (Hrsg.) (2010a). TEDS-M 2008 – Professionelle Kompetenz und Lerngelegenheiten angehender Primarstufenlehrkräfte im internationalen Vergleich. Münster: Waxmann.
Blömeke, S., Kaiser, G., & Lehmann, R. (Hrsg.) (2010b). TEDS-M 2008 – Professionelle Kompetenz und Lerngelegenheiten angehender Mathematiklehrkräfte im internationalen Vergleich. Münster: Waxmann.
Klieme, E. & Leutner, D. (2006). Kompetenzmodelle zur Erfassung individueller Lernergebnisse und zur Bilanzierung von Bildungsprozessen. Zeitschrift für Pädagogik, 6, 876-903.König, J. & Blömeke, S. (2009c). Disziplin- oder Berufsorientierung? Zur Struktur des pädagogischen Wissens angehender Lehrkräfte. Lehrerbildung auf dem Prüfstand, 2 (1), 126-147.
König, J. , Blömeke, S., Paine, L., Schmidt, B. & Feng-Jui Hsieh (2011). General Pedagogical Knowledge of Future Middle School Teachers. On the Complex Ecology of Teacher Education in the United States, Germany, and Taiwan. Journal of Teacher Education, 62 (2), 188-201.
Oelkers, J. & Tenorth, H.-E. (Hrsg.) (1993). Pädagogisches Wissen. Weinheim: Beltz.
Schaefers, C. (2002). Forschung zur Lehrerausbildung in Deutschland - eine bilanzierende Übersicht der neueren empirischen Studien. Schweizerische Zeitschrift für Bildungswissenschaften, 24, 65-88.
Seifert, A. & Schaper, N. (2010). Überprüfung eines Kompetenzmodells und Messinstruments zur Strukturierung allgemeiner pädagogischer Kompetenz in der universitären Lehrerbildung. Lehrerbildung auf dem Prüfstand, 3 (2), 179-198.
Seifert, A., Hilligus, A. H., & Schaper, N. (2009). Entwicklung und psychometrische Überprüfung eines Messinstrumentes zur Erfassung pädagogischer Kompetenzen in der universitären Lehrerbildung. Lehrerbildung auf dem Prüfstand, 2 (1), 82-103.
Tenorth, H.-E. (2006). Professionalität im Lehrerberuf. Ratlosigkeit der Theorie, gelingende Praxis. Zeitschrift für Erziehungswissenschaft, 9, 580-597.
Terhart, E. (Ed.) (2000). Perspektiven der Lehrerbildung in Deutschland. Abschlussbericht der von der Kultusministerkonferenz eingesetzten Kommission. Weinheim: Beltz.
Terhart, E. (2002). Standards für die Lehrerbildung. Eine Expertise für die Kultusministerkonferenz. Münster: Universität Münster.
Wu, M. L., Adams, R. J., & Wilson, M. R. (1997). ConQuest: Multi-Aspect Test Software [Computer program]. Camberwell, Vic.: Australian Council for Educational Research.
Three questions to be discussed with the audience
1. How can the instruments of general pedagogical knowledge be criterion-validated (e.g. rating the quality of teacher behavior in videos of lessons, or measuring stress hormone level after class)?
2. Are the operationalizations of general pedagogical knowledge valid in other (European) nations?
3. Is general pedagogical knowledge more or less important than content knowledge to be a successful and professional teacher?
13.06.2012
Event
EARLI SIG 11 Conference 2012: Teaching and Teacher Education in an Era of Accountability – What do we know and what do we need to know?
13.06.12 → 15.06.12
Bergen, NorwayEvent: Conference
- Empirical education research