Construct relation extraction from scientific papers: Is it automatable yet?

Research output: Contributions to collected editions/worksPublished abstract in conference proceedingsResearchpeer-review

Authors

The process of identifying relevant prior research articles is crucial for theoretical advancements, but often requires significant human effort. This study examines the feasibility of using large language models (LLMs) to support this task by extracting tested hypotheses, which consist of related constructs, moderators or mediators, path coefficients, and p-values, from empirical studies using structural equation modeling (SEM). We combine state-of-the-art LLMs with a variety of post-processing measures to improve the relation extraction quality. An extensive evaluation yields recall scores of up to 79.2% in construct entity extraction, 58.4% in construct-mediator/moderator-construct extraction, and 39.3% in extracting the full tested hypotheses. We provide a manually annotated dataset of 72 SEM articles and 749 construct relations to facilitate future research. Our findings offer critical insights and suggest promising directions for advancing the field of automated construct relation extraction from scholarly documents.
Original languageEnglish
Title of host publicationProceedings of the 58th Hawaii International Conference on System Sciences
EditorsTung Bui
Number of pages10
Place of PublicationHonolulu
PublisherUniversity of Hawaii at Manoa
Publication date07.01.2025
Pages4675-4684
ISBN (electronic)978-0-9981331-8-8
Publication statusPublished - 07.01.2025
Event58th Hawaii International Conference on System Sciences - HICSS 2025 - Hilton Waikoloa Village, Waikoloa, United States
Duration: 07.01.202510.01.2025
Conference number: 58

Bibliographical note

Collections: AI Assistants and Generative AI for Knowledge Creation, Retention, and Use

    Research areas

  • Business informatics - AI Assistants and Generative AI for Knowledge Creation, Retention, and Use, Large Language Models, natural language processing, Relation extraction, structural equation modeling

Links

Recently viewed

Activities

  1. Einkommensdynamik von Freien Berufen und Selbstständigen
  2. Exploring Team Cohesion Through Humour. An Ethnographic Study of a Professional Football Team
  3. Tagung "Radioforschung/Audioforschung II
  4. Conceptualizing the Governance of Global Telecoupling
  5. Universität von Westengland
  6. ENTER Conference 2011
  7. Ziele
  8. Networking workshop Helmholtz-Zentrum Geesthacht -HHG 2013
  9. Machbarkeitsstudie zu einem Visitor Welcome Center auf Usedom
  10. Indonesia Global Entrepreneurship Network
  11. Going Green 2015 - Exploring Sustainability in Virtual Space
  12. The Contagion Effect of Populism on Mainstream Parties
  13. Out of Control - Der performative Spielraum in Kunst und Kybernetik - 2013
  14. Nachhaltiger Konsum in der Wachstumsgesellschaft
  15. Transportphänomene in dynamischen Systemen
  16. Mitglied des IFIP-Beirats, Sitzung des IFIP-Beirats
  17. Fakultät MT allgemein (Organisation)
  18. Marketing of Public and Non-Profit Organisations Understanding Audiences and how to Persuade them in Public and NPO
  19. SASE 2020 Annual Conference
  20. Panel session on “The grand challenges of the 21st century”
  21. 39th International Conference on Improving University Teaching - IUT 2014
  22. Workshop der Forschungsgruppe "Computersignale"
  23. Wissensintegration und Wissenstransfer in der transdisziplinären Nachhaltigkeitsforschung: Das Beispiel Global TraPs.
  24. Dispositive Funktionaler Klänge - 2015
  25. 9th International Federation on Automatic Control Workshop on Intelligent Manufacturing Systems - 2008
  26. 22nd Annual Graduate Student Conference - 2020
  27. International Journal of Innovation and Sustainable Development (Zeitschrift)
  28. Tech and TEFL - Learning and Teaching English in the Digital Age
  29. A Hybrid Extended Kalman Filter as an Observer for a Pot-Electro-Magnetic Actuator
  30. Utopie-Konferenz 2018
  31. Evaluator of the PhD Thesis of Eduardo Moya Lasheras and member of the commission of the final examination
  32. On the sovereignty over one’s personal data
  33. INMATCO-Projekt: Interest in Mathematical Task Context. Von VAMPS zu INMATCO - Geschichte eines psychologisch-mathematikdidaktischen Forschungsprojekts
  34. Concept Maps - Eine Einführung zur Strukturierung und Visualisierung von Wissen mit Hilfe von Cmap Tools 2009