Surveying the FAIRness of Annotation Tools: Difficult to find, difficult to reuse

Research output: Contributions to collected editions/worksArticle in conference proceedingsResearchpeer-review

Authors

  • Ekaterina Borisova
  • Raia Abu Ahmad
  • Leyla Jael Garcia-Castro
  • Ricardo Usbeck
  • Georg Rehm
In the realm of Machine Learning and Deep Learning, there is a need for high-quality annotated data to train and evaluate supervised models. An extensive number of annotation tools have been developed to facilitate the data labelling process. However, finding the right tool is a demanding task involving thorough searching and testing. Hence, to effectively navigate the multitude of tools, it becomes essential to ensure their findability, accessibility, interoperability, and reusability (FAIR). This survey addresses the FAIRness of existing annotation software by evaluating 50 different tools against the FAIR principles for research software (FAIR4RS). The study indicates that while being accessible and interoperable, annotation tools are difficult to find and reuse. In addition, there is a need to establish community standards for annotation software development, documentation, and distribution.
Original languageEnglish
Title of host publicationLAW 2024 - 18th Linguistic Annotation Workshop, Co-located with EACL 2024 - Proceedings of the Workshop : Proceedings of the Workshop
EditorsSophie Henning, Manfred Stede
Number of pages17
Place of PublicationStroudsburg
PublisherAssociation for Computational Linguistics (ACL)
Publication date01.03.2024
Pages29-45
ISBN (electronic)979-8-89176-073-8
Publication statusPublished - 01.03.2024
Event18th Linguistic Annotation Workshop - St. Julians, Malta
Duration: 21.03.202422.03.2024
Conference number: 18
https://www.aclweb.org/portal/content/first-call-papers-18th-linguistic-annotation-workshop

Bibliographical note

Publisher Copyright:
© 2024 Association for Computational Linguistics.

Links

Recently viewed

Publications

  1. The effect of complacency potential on human operators’ monitoring behavior in aviation
  2. INFLEXIBLE COMMODITY PRICES AS A RESULT OF PROFIT MAXIMIZATION - REPLY TO A COMMENT BY MAUSSNER,A.
  3. How Participatory Should Environmental Governance Be?
  4. Comment on "Recent origin and cultural reversion of a hunter-gatherer group
  5. Using meaningful places as an indicator for sense of place in the management of social-ecological systems
  6. Group membership does not modulate automatic imitation
  7. Klimapaket
  8. "God Is in the Details", or The Filing Box Answers
  9. Introduction
  10. Germany
  11. Vergesellschaftung durch Konsum
  12. Public understanding of climate change terminology in Germany
  13. More than Yield
  14. Verkleben
  15. Einleitung
  16. Bioconversion of agri-food residues into lactic acid
  17. Influence of three different balance shoes on EMG-activity during one-leg standing
  18. Impact and Importance of Heterocyclics in Remediation
  19. Materialities of the Performative
  20. Towards a decision support system for radiotherapy business continuity in a pandemic crisis
  21. Editors’ Conversation with German Art Historians Oona Lochner and Isabel Mehl: Writing Like a Feminist—In Dialogue with Carla Lonzi
  22. Emotional design and positive emotions in multimedia learning
  23. Alex Adriaansens
  24. Omega
  25. Einsame Lektüre oder Kommunikation?
  26. Mehrsprachigkeit und Interkulturalität in Fremdsprachenportfolios
  27. Christian Boltanski's Failed Meta-Archive
  28. Diskussionsforum
  29. Using ectomycorrhizae to improve the restoration of Neotropical coastal zones
  30. Wenig Lärm um nichts