Surveying the FAIRness of Annotation Tools: Difficult to find, difficult to reuse

Research output: Contributions to collected editions/worksArticle in conference proceedingsResearchpeer-review

Authors

  • Ekaterina Borisova
  • Raia Abu Ahmad
  • Leyla Jael Garcia-Castro
  • Ricardo Usbeck
  • Georg Rehm
In the realm of Machine Learning and Deep Learning, there is a need for high-quality annotated data to train and evaluate supervised models. An extensive number of annotation tools have been developed to facilitate the data labelling process. However, finding the right tool is a demanding task involving thorough searching and testing. Hence, to effectively navigate the multitude of tools, it becomes essential to ensure their findability, accessibility, interoperability, and reusability (FAIR). This survey addresses the FAIRness of existing annotation software by evaluating 50 different tools against the FAIR principles for research software (FAIR4RS). The study indicates that while being accessible and interoperable, annotation tools are difficult to find and reuse. In addition, there is a need to establish community standards for annotation software development, documentation, and distribution.
Original languageEnglish
Title of host publicationLAW 2024 - 18th Linguistic Annotation Workshop, Co-located with EACL 2024 - Proceedings of the Workshop : Proceedings of the Workshop
EditorsSophie Henning, Manfred Stede
Number of pages17
Place of PublicationStroudsburg
PublisherAssociation for Computational Linguistics (ACL)
Publication date01.03.2024
Pages29-45
ISBN (electronic)979-8-89176-073-8
Publication statusPublished - 01.03.2024
Event18th Linguistic Annotation Workshop - St. Julians, Malta
Duration: 21.03.202422.03.2024
Conference number: 18
https://www.aclweb.org/portal/content/first-call-papers-18th-linguistic-annotation-workshop

Bibliographical note

Publisher Copyright:
© 2024 Association for Computational Linguistics.

Links

Recently viewed

Publications

  1. Effekte fachdidaktischer Ansätze auf den Schriftspracherwerb in der Grundschule
  2. Gesellschaftliche Funktionen von Museen zwischen Assimilation und Akkommodation
  3. Versuch zu Formen von Öffentlichkeit in lokalen Bildungs- und Sorgelandschaften
  4. Wandel des Staates als Herausforderung für die Didaktik der Politischen Bildung
  5. Neustrukturierung des Mittleren Freihafens zum Central Terminal Steinwerder (CTS)
  6. Stakeholder involvement for Water Framework Directive implementation in Germany
  7. The positive effects of a protean career attitude for self-initiated expatriates
  8. Die "Leute" und die "Kunst" - eine "Heisenberg'sche Unschärferelation" der Stadtplanung
  9. What You Sow Is What You Reap? (Dis-)Incentives for Adaptation Intentions in Farming
  10. The shooter bias: Replicating the classic effect and introducing a novel paradigm
  11. Die Kehrseite der Medaille. Popkultur und «Subversion» im Kontext der Neuen Rechten
  12. Monetäre Bewertung von Erholungsnutzen und Nutzerkonflikten in stadtnahen Wäldern
  13. Underestimation of personal carbon footprint inequality in four diverse countries
  14. Globales Lernen und Biodiversität in der außerschulischen Bildung in Deutschland
  15. Handbuch liturgische Präsenz, Bd. 1, Zur praktischen Inszenierung des Gottesdienstes
  16. Failure to Learn From Failure Is Mitigated by Loss-Framing and Corrective Feedback
  17. Development and Testing of Water-Filled Tube Systems for Flood Protection Measures
  18. Perspektiven der Teilwertabschreibung bei voraussichtlich dauernder Wertminderung
  19. Sustainability economics - General versus specific, and conceptual versus practical
  20. Priority effects caused by plant order of arrival affect below-ground productivity
  21. Techniques to Control Microbial Contaminants in Nonsterile Microalgae Cultivation
  22. Parlamentarische Informations- und Kontrollressourcen in 22 westlichen Demokratien
  23. Antirassistischer Widerstand von Migrantinnen und Migranten in der Bundesrepublik
  24. Exports, R&D and Productivity: A test of the Bustos-model with German enterprise data
  25. Thermal energy storage systems for heating and hot water in residential buildings
  26. The multiplicity of emotions: A framework of emotional functions in decision making
  27. "to expose, to show, to demonstrate, to inform, to offer. Artistic Practices around 1990"
  28. Do Serious Breaches Give Rise to Any Specific Obligations of the Responsible State?
  29. Gender Issues im Internationalen Studiengang "Integrated Water Resources Management"
  30. Strategic control over the unhelpful effects of primed social categories and goals
  31. Coauthoring collaborative strategy when voices are many and authority is ambiguous
  32. Solid Sorption Refrigeration With Calcium Chloride Methanolates on Technical Scale
  33. Die Mutter-Verantwortlichkeit für Menschenrechtsverletzungen ihrer Auslandstöchter
  34. What do we know about meta-analyses in accounting, auditing, and corporate governance?
  35. Introduction: Two Centuries of the Sublime in American Landscape, Art, and Literature
  36. Cross-border venture capital flows and local ties: Evidence from developed countries
  37. Internetbasierte Finanzkommunikation bei der Unternehmensfinanzierung mit Anleihen