GERBIL - General entity annotator benchmarking framework

Publikation: Beiträge in SammelwerkenAufsätze in KonferenzbändenForschungbegutachtet

Authors

  • Michael Röder
  • Axel Cyrille Ngonga Ngomo
  • Ciro Baron
  • Andreas Both
  • Martin Brümmer
  • Diego Ceccarelli
  • Marco Cornolti
  • Didier Cherix
  • Bernd Eickmann
  • Paolo Ferragina
  • Christiane Lemke
  • Andrea Moro
  • Roberto Navigli
  • Francesco Piccinno
  • Giuseppe Rizzo
  • Harald Sack
  • René Speck
  • Raphaël Troncy
  • Jörg Waitelonis
  • Lars Wesemann

The need to bridge between the unstructured data on the Document Web and the structured data on the Web of Data has led to the development of a considerable number of annotation tools. However, these tools are currently still hard to compare since the published evaluation results are calculated on diverse datasets and evaluated based on different measures. We present GERBIL, an evaluation framework for semantic entity annotation. The rationale behind our framework is to provide developers, end users and researchers with easy-To-use interfaces that allow for the agile, fine-grained and uniform evaluation of annotation tools on multiple datasets. By these means, we aim to ensure that both tool developers and end users can derive meaningful insights pertaining to the extension, integration and use of annotation applications. In particular, GERBIL provides comparable results to tool developers so as to allow them to easily discover the strengths and weaknesses of their implementations with respect to the state of the art. With the permanent experiment URIs provided by our framework, we ensure the reproducibility and archiving of evaluation results. Moreover, the framework generates data in machineprocessable format, allowing for the efficient querying and post-processing of evaluation results. Finally, the tool diagnostics provided by GERBIL allows deriving insights pertaining to the areas in which tools should be further refined, thus allowing developers to create an informed agenda for extensions and end users to detect the right tools for their purposes. GERBIL aims to become a focal point for the state of the art, driving the research agenda of the community by presenting comparable objective evaluation results.

OriginalspracheEnglisch
TitelWWW 2015 - Proceedings of the 24th International Conference on World Wide Web
HerausgeberAldo Gangemi, Stefano Leonardi, Alessandro Panconesi
Anzahl der Seiten11
VerlagAssociation for Computing Machinery, Inc
Erscheinungsdatum18.05.2015
Seiten1133-1143
ISBN (Print)978-1-4503-3469-3
DOIs
PublikationsstatusErschienen - 18.05.2015
Extern publiziertJa
Veranstaltung24th International Conference on World Wide Web, WWW 2015 - Florence, Italien
Dauer: 18.05.201522.05.2015
https://dl.acm.org/doi/proceedings/10.1145/2740908
https://dblp.org/db/conf/www/www2015.html

Links

DOI

Zuletzt angesehen

Aktivitäten

  1. Enlightened Mythology.: Thomas Mann and Myth
  2. INTERMOLECULAR SPACES 2016
  3. Transdisciplinary Evaluation of Different Coastal Adaptation Strategies: Integrating Regional Perceptions of Scientists, Practitioners and the Public
  4. Are cities mediators of uncertainty in the field of deportation?
  5. Time and Income Poverty Dynamics - An Interdependent Multidimensional Approach with German Time Use Data
  6. Workshop: The conceptualization of sound within a studio environment 2011
  7. Are Self-Employed Time and Money Poor? Dynamics of Interpendent Multidimensional Poverty with German Time Use Diary Data
  8. Student Consumer Learning in Schools‘ „Cultures of Consumption“: Results of an Empirical Study
  9. Legitimizing Digital Transformation within an Incumbent: How Unaccepted Leaders Can Initiate Strategic Changes
  10. Automatic Detection and Classification of State Heads and Common People?
  11. The power and peril of precise numbers
  12. Measurement of Mental Models of Climate Change. Cross-Linked Thinking through Scenario Analysis
  13. Paper: The Multilevel Governance of Deportation – is Access to local Membership possible?
  14. Comparative Perspectives on Social Values and Modernization - 2011
  15. Karlstad Universität
  16. The Ethics of Memory - 2009
  17. Picasso and AI: analysing and questioning the technology that overwhelms us
  18. Who does take part in which kind of teacher professional development?
  19. Wien Depot: Podiumsdiskussion
  20. PhD Workshop on Theories in Information Systems Science
  21. Spas in the New Länder: A Transformation with an Uncertain Outcome.
  22. Interdisziplinarität in kuratorischen Netzwerken
  23. CHANGE Expertenworkshop 2008
  24. Education for Sustainable Development – Experiences from Theory and Practice
  25. Stimmtraining - 2009
  26. Towards a New Aesthetic Paradigm
  27. HyperKult XXIII – Bring your own... 2014
  28. Karlstad Universität

Presse / Medien

  1. Performance unter Hypnose