A Meta-Analytical Multilevel Reliability Generalization of Situational Judgment Tests (SJTs)
Research output: Journal contributions › Journal articles › Research › peer-review
Standard
In: European Journal of Psychological Assessment, Vol. 32, No. 3, 07.2016, p. 230-240.
Research output: Journal contributions › Journal articles › Research › peer-review
Harvard
APA
Vancouver
Bibtex
}
RIS
TY - JOUR
T1 - A Meta-Analytical Multilevel Reliability Generalization of Situational Judgment Tests (SJTs)
AU - Kasten, Nadine
AU - Freund, Philipp Alexander
PY - 2016/7
Y1 - 2016/7
N2 - During the past 20 years, Situational Judgment Tests (SJTs) have developed into a viable tool in personnel selection. Despite their growing popularity, research examining the extent of measurement error is widely lacking. Using reliability generalization, the aim of this article was twofold: (1) establish an estimate for an average coefficient alpha of SJT scores across studies and (2) examine the influence of essential SJT features and selected study variables on score reliability. To handle potential dependent observations a three-level hierarchical linear model was used. The results indicate that the reliability of SJT scores is typically rather low and below recommended levels for highstakes applications. Additionally, both SJT and study characteristics affect score reliability. Implications for practitioners and researchers are provided to guide an appropriate use of SJTs and to initiate future research.
AB - During the past 20 years, Situational Judgment Tests (SJTs) have developed into a viable tool in personnel selection. Despite their growing popularity, research examining the extent of measurement error is widely lacking. Using reliability generalization, the aim of this article was twofold: (1) establish an estimate for an average coefficient alpha of SJT scores across studies and (2) examine the influence of essential SJT features and selected study variables on score reliability. To handle potential dependent observations a three-level hierarchical linear model was used. The results indicate that the reliability of SJT scores is typically rather low and below recommended levels for highstakes applications. Additionally, both SJT and study characteristics affect score reliability. Implications for practitioners and researchers are provided to guide an appropriate use of SJTs and to initiate future research.
KW - Psychology
KW - Cronbach's alpha
KW - Reliability generalization
KW - Situational Judgment Test
KW - SJT
UR - http://www.scopus.com/inward/record.url?scp=84989897154&partnerID=8YFLogxK
U2 - 10.1027/1015-5759/a000250
DO - 10.1027/1015-5759/a000250
M3 - Journal articles
VL - 32
SP - 230
EP - 240
JO - European Journal of Psychological Assessment
JF - European Journal of Psychological Assessment
SN - 1015-5759
IS - 3
ER -