TY - CHAP

T1 - Construct relation extraction from scientific papers

T2 - 58th Hawaii International Conference on System Sciences - HICSS 2025

AU - Funk, Burkhardt

AU - Scharfenberger, Jonas

N1 - Conference code: 58

PY - 2025/1/7

Y1 - 2025/1/7

N2 - The process of identifying relevant prior research articles is crucial for theoretical advancements, but often requires significant human effort. This study examines the feasibility of using large language models (LLMs) to support this task by extracting tested hypotheses, which consist of related constructs, moderators or mediators, path coefficients, and p-values, from empirical studies using structural equation modeling (SEM). We combine state-of-the-art LLMs with a variety of post-processing measures to improve the relation extraction quality. An extensive evaluation yields recall scores of up to 79.2% in construct entity extraction, 58.4% in construct-mediator/moderator-construct extraction, and 39.3% in extracting the full tested hypotheses. We provide a manually annotated dataset of 72 SEM articles and 749 construct relations to facilitate future research. Our findings offer critical insights and suggest promising directions for advancing the field of automated construct relation extraction from scholarly documents.

AB - The process of identifying relevant prior research articles is crucial for theoretical advancements, but often requires significant human effort. This study examines the feasibility of using large language models (LLMs) to support this task by extracting tested hypotheses, which consist of related constructs, moderators or mediators, path coefficients, and p-values, from empirical studies using structural equation modeling (SEM). We combine state-of-the-art LLMs with a variety of post-processing measures to improve the relation extraction quality. An extensive evaluation yields recall scores of up to 79.2% in construct entity extraction, 58.4% in construct-mediator/moderator-construct extraction, and 39.3% in extracting the full tested hypotheses. We provide a manually annotated dataset of 72 SEM articles and 749 construct relations to facilitate future research. Our findings offer critical insights and suggest promising directions for advancing the field of automated construct relation extraction from scholarly documents.

KW - Business informatics

KW - AI Assistants and Generative AI for Knowledge Creation

KW - Retention, and Use

KW - Large Language Models

KW - natural language processing

KW - Relation extraction

KW - structural equation modeling

M3 - Published abstract in conference proceedings

T3 - Hawaii International Conference on System Sciences (HICSS)

SP - 4675

EP - 4684

BT - Proceedings of the 58th Hawaii International Conference on System Sciences

A2 - Bui, Tung

PB - University of Hawaiʻi at Mānoa

CY - Honolulu

Y2 - 7 January 2025 through 10 January 2025

ER -