Surveying the FAIRness of Annotation Tools: Difficult to find, difficult to reuse

Publikation: Beiträge in SammelwerkenAufsätze in KonferenzbändenForschungbegutachtet

Standard

Surveying the FAIRness of Annotation Tools: Difficult to find, difficult to reuse. / Borisova, Ekaterina; Abu Ahmad, Raia; Garcia-Castro, Leyla Jael et al.
LAW 2024 - 18th Linguistic Annotation Workshop, Co-located with EACL 2024 - Proceedings of the Workshop: Proceedings of the Workshop. Hrsg. / Sophie Henning; Manfred Stede. Stroudsburg: Association for Computational Linguistics (ACL), 2024. S. 29-45.

Publikation: Beiträge in SammelwerkenAufsätze in KonferenzbändenForschungbegutachtet

Harvard

Borisova, E, Abu Ahmad, R, Garcia-Castro, LJ, Usbeck, R & Rehm, G 2024, Surveying the FAIRness of Annotation Tools: Difficult to find, difficult to reuse. in S Henning & M Stede (Hrsg.), LAW 2024 - 18th Linguistic Annotation Workshop, Co-located with EACL 2024 - Proceedings of the Workshop: Proceedings of the Workshop. Association for Computational Linguistics (ACL), Stroudsburg, S. 29-45, The 18th Linguistic Annotation Workshop, St. Julians, Malta, 21.03.24. <https://aclanthology.org/2024.law-1.4>

APA

Borisova, E., Abu Ahmad, R., Garcia-Castro, L. J., Usbeck, R., & Rehm, G. (2024). Surveying the FAIRness of Annotation Tools: Difficult to find, difficult to reuse. In S. Henning, & M. Stede (Hrsg.), LAW 2024 - 18th Linguistic Annotation Workshop, Co-located with EACL 2024 - Proceedings of the Workshop: Proceedings of the Workshop (S. 29-45). Association for Computational Linguistics (ACL). https://aclanthology.org/2024.law-1.4

Vancouver

Borisova E, Abu Ahmad R, Garcia-Castro LJ, Usbeck R, Rehm G. Surveying the FAIRness of Annotation Tools: Difficult to find, difficult to reuse. in Henning S, Stede M, Hrsg., LAW 2024 - 18th Linguistic Annotation Workshop, Co-located with EACL 2024 - Proceedings of the Workshop: Proceedings of the Workshop. Stroudsburg: Association for Computational Linguistics (ACL). 2024. S. 29-45

Bibtex

@inbook{9b276e61c76544348064c83d2430071c,
title = "Surveying the FAIRness of Annotation Tools: Difficult to find, difficult to reuse",
abstract = "In the realm of Machine Learning and Deep Learning, there is a need for high-quality annotated data to train and evaluate supervised models. An extensive number of annotation tools have been developed to facilitate the data labelling process. However, finding the right tool is a demanding task involving thorough searching and testing. Hence, to effectively navigate the multitude of tools, it becomes essential to ensure their findability, accessibility, interoperability, and reusability (FAIR). This survey addresses the FAIRness of existing annotation software by evaluating 50 different tools against the FAIR principles for research software (FAIR4RS). The study indicates that while being accessible and interoperable, annotation tools are difficult to find and reuse. In addition, there is a need to establish community standards for annotation software development, documentation, and distribution.",
keywords = "Business informatics",
author = "Ekaterina Borisova and {Abu Ahmad}, Raia and Garcia-Castro, {Leyla Jael} and Ricardo Usbeck and Georg Rehm",
note = "Publisher Copyright: {\textcopyright} 2024 Association for Computational Linguistics.; 18th Linguistic Annotation Workshop , LAW-XVIII ; Conference date: 21-03-2024 Through 22-03-2024",
year = "2024",
month = mar,
day = "1",
language = "English",
pages = "29--45",
editor = "Sophie Henning and Manfred Stede",
booktitle = "LAW 2024 - 18th Linguistic Annotation Workshop, Co-located with EACL 2024 - Proceedings of the Workshop",
publisher = "Association for Computational Linguistics (ACL)",
address = "United States",
url = "https://www.aclweb.org/portal/content/first-call-papers-18th-linguistic-annotation-workshop",

}

RIS

TY - CHAP

T1 - Surveying the FAIRness of Annotation Tools: Difficult to find, difficult to reuse

AU - Borisova, Ekaterina

AU - Abu Ahmad, Raia

AU - Garcia-Castro, Leyla Jael

AU - Usbeck, Ricardo

AU - Rehm, Georg

N1 - Conference code: 18

PY - 2024/3/1

Y1 - 2024/3/1

N2 - In the realm of Machine Learning and Deep Learning, there is a need for high-quality annotated data to train and evaluate supervised models. An extensive number of annotation tools have been developed to facilitate the data labelling process. However, finding the right tool is a demanding task involving thorough searching and testing. Hence, to effectively navigate the multitude of tools, it becomes essential to ensure their findability, accessibility, interoperability, and reusability (FAIR). This survey addresses the FAIRness of existing annotation software by evaluating 50 different tools against the FAIR principles for research software (FAIR4RS). The study indicates that while being accessible and interoperable, annotation tools are difficult to find and reuse. In addition, there is a need to establish community standards for annotation software development, documentation, and distribution.

AB - In the realm of Machine Learning and Deep Learning, there is a need for high-quality annotated data to train and evaluate supervised models. An extensive number of annotation tools have been developed to facilitate the data labelling process. However, finding the right tool is a demanding task involving thorough searching and testing. Hence, to effectively navigate the multitude of tools, it becomes essential to ensure their findability, accessibility, interoperability, and reusability (FAIR). This survey addresses the FAIRness of existing annotation software by evaluating 50 different tools against the FAIR principles for research software (FAIR4RS). The study indicates that while being accessible and interoperable, annotation tools are difficult to find and reuse. In addition, there is a need to establish community standards for annotation software development, documentation, and distribution.

KW - Business informatics

UR - https://aclanthology.org/2024.law-1.pdf

UR - http://www.scopus.com/inward/record.url?scp=85188717570&partnerID=8YFLogxK

M3 - Article in conference proceedings

SP - 29

EP - 45

BT - LAW 2024 - 18th Linguistic Annotation Workshop, Co-located with EACL 2024 - Proceedings of the Workshop

A2 - Henning, Sophie

A2 - Stede, Manfred

PB - Association for Computational Linguistics (ACL)

CY - Stroudsburg

T2 - 18th Linguistic Annotation Workshop

Y2 - 21 March 2024 through 22 March 2024

ER -

Links