Avoiding algorithm errors in textual analysis: A guide to selecting software, and a research agenda toward generative artificial intelligence
Research output: Journal contributions › Journal articles › Research › peer-review
Standard
In: Journal of Business Research, Vol. 199, 115571, 10.2025.
Research output: Journal contributions › Journal articles › Research › peer-review
Harvard
APA
Vancouver
Bibtex
}
RIS
TY - JOUR
T1 - Avoiding algorithm errors in textual analysis
T2 - A guide to selecting software, and a research agenda toward generative artificial intelligence
AU - Wobst, Janice
AU - Lueg, Rainer
N1 - Publisher Copyright: © 2025 The Authors
PY - 2025/10
Y1 - 2025/10
N2 - The use of textual analysis is expanding in organizational research, yet software packages vary in their compatibility with complex constructs. This study helps researchers select suitable tools by focusing on phrase-based dictionary methods. We empirically evaluate four software packages—LIWC, DICTION, CAT Scanner, and a custom Python tool—using the complex construct of value-based management as a test case. The analysis shows that software from the same methodological family produces highly consistent results, while popular but mismatched tools yield significant errors such as miscounted phrases. Based on this, we develop a structured selection guideline that links construct features with software capabilities. The framework enhances construct validity, supports methodological transparency, and is applicable across disciplines. Finally, we position the approach as a bridge to AI-enabled textual analysis, including prompt-based workflows, reinforcing the continued need for theory-grounded construct design.
AB - The use of textual analysis is expanding in organizational research, yet software packages vary in their compatibility with complex constructs. This study helps researchers select suitable tools by focusing on phrase-based dictionary methods. We empirically evaluate four software packages—LIWC, DICTION, CAT Scanner, and a custom Python tool—using the complex construct of value-based management as a test case. The analysis shows that software from the same methodological family produces highly consistent results, while popular but mismatched tools yield significant errors such as miscounted phrases. Based on this, we develop a structured selection guideline that links construct features with software capabilities. The framework enhances construct validity, supports methodological transparency, and is applicable across disciplines. Finally, we position the approach as a bridge to AI-enabled textual analysis, including prompt-based workflows, reinforcing the continued need for theory-grounded construct design.
KW - Algorithm error
KW - Generative AI
KW - Large language models
KW - Reliability
KW - Software selection
KW - Textual analysis
KW - Validity
KW - Value-based management
KW - Management studies
UR - http://www.scopus.com/inward/record.url?scp=105009249410&partnerID=8YFLogxK
U2 - 10.1016/j.jbusres.2025.115571
DO - 10.1016/j.jbusres.2025.115571
M3 - Journal articles
AN - SCOPUS:105009249410
VL - 199
JO - Journal of Business Research
JF - Journal of Business Research
SN - 0148-2963
M1 - 115571
ER -