Enhancing careless responding detection: A norm group-based calculation approach

Activity: Talk or presentationConference PresentationsResearch

Yannik A. Escher - Speaker

Philippe Goldammer - Coauthor

Peter L. Stöckli - Coauthor

Hubert Annen - Coauthor

Anna Luca Heimann - Coauthor

Undetected careless responding in survey data poses a significant threat to the validity of psychological research and questionnaire-based decisions. Even small portions of careless responses can bias reliability estimates, factor loadings, or item means, ultimately distorting findings from widely used Likert scale-based questionnaires. Across two pre-registered studies, this research investigates if and under what conditions a new set of norm group-based indices outperforms promising conventional methods in detecting careless respondents (i.e., Mahalanobis Distance and person-total/personal-biserial correlation coefficient). In Study 1, we will simulate n = 500 observations across k = 100 replications and m = 72 conditions, resulting in a total of 7,200 samples. Additionally, in Study 2, we will conduct a re-analysis of real incentivized data (N = 481), using our proposed implicit norm group-based indices. Data collection for both studies is ongoing, and results will be presented at AOWI 2025. We will discuss potential limitations, such as parameter constraints in Study 1 (e.g., careless patterns, norm group sizes) or confounding factors in Study 2 (e.g., survey length, the presence of genuinely careless respondents). The present study will provide practical guidance for improving the detection of careless responding in survey-based research and offers a novel approach for calculating implicit careless responding indices. This work will contribute to more reliable survey-based assessments in organizational psychology and hopefully enhance the credibility of psychological findings derived from questionnaires and surveys.
12.09.2025

Event

14. Fachgruppentagung der Arbeits-, Organisations- und Wirtschaftspsychologie und Ingenieurspsychologie - AOWI 2025: Arbeitswelten im Wandel: Kooperation zwischen Mensch und Technik

09.09.2512.09.25

Lüneburg, Lower Saxony, Germany

Event: Conference