TY - JOUR

T1 - Avoiding Algorithm Error in Computer-Aided Text Analyses

T2 - 82nd Annual Meeting of the Academy of Management - AOM 2022

AU - Wobst, Janice

AU - Lueg, Rainer

N1 - Conference code: 82

PY - 2022/8/1

Y1 - 2022/8/1

N2 - The use of computer-aided text analysis as a tool to quantify qualitative information continues to grow in management research. Academia increasingly develops guiding principles that aid scholars in addressing reliability and validity issues. However, there is little guidance in selecting a proper software solution that aligns the features of the software with the features of the underlying construct. Further guidelines are particularly relevant because computer-aided text analysis is highly software dependent. This paper demonstrates a systematic software selection process using two illustrative constructs: a theoretically complex construct (value- based management sophistication) and a more straightforward construct (firm’s future orientation). The paper develops generalizable principles for selecting proper software solutions. After reviewing relevant literature, this paper compares the algorithmic proximity of frequently applied software solutions in management research, performing analysis of variance and convergent validity analysis. The results demonstrate that the algorithmic performance depends on the construct type. The analysis of variance indicates statistically significant differences in mean values between various software solutions. However, these differences do not lead to substantial deviations in convergent validity results between the applied software solutions. This paper contributes to the literature by assisting scholars in choosing a proper software solution when conducting computer-aided text analysis.

AB - The use of computer-aided text analysis as a tool to quantify qualitative information continues to grow in management research. Academia increasingly develops guiding principles that aid scholars in addressing reliability and validity issues. However, there is little guidance in selecting a proper software solution that aligns the features of the software with the features of the underlying construct. Further guidelines are particularly relevant because computer-aided text analysis is highly software dependent. This paper demonstrates a systematic software selection process using two illustrative constructs: a theoretically complex construct (value- based management sophistication) and a more straightforward construct (firm’s future orientation). The paper develops generalizable principles for selecting proper software solutions. After reviewing relevant literature, this paper compares the algorithmic proximity of frequently applied software solutions in management research, performing analysis of variance and convergent validity analysis. The results demonstrate that the algorithmic performance depends on the construct type. The analysis of variance indicates statistically significant differences in mean values between various software solutions. However, these differences do not lead to substantial deviations in convergent validity results between the applied software solutions. This paper contributes to the literature by assisting scholars in choosing a proper software solution when conducting computer-aided text analysis.

KW - Management studies

UR - https://www.mendeley.com/catalogue/b0111894-6344-37b7-8715-2fcc9981fbb5/

U2 - 10.5465/AMBPP.2022.10504abstract

DO - 10.5465/AMBPP.2022.10504abstract

M3 - Conference abstract in journal

VL - 2022

JO - Academy of Management Proceedings

JF - Academy of Management Proceedings

SN - 0065-0668

IS - 1

Y2 - 5 August 2022 through 9 August 2022

ER -