Hands in Focus: Sign Language Recognition Via Top-Down Attention

Research output: Contributions to collected editions/worksArticle in conference proceedingsResearchpeer-review

Authors

  • Noha Sarhan
  • Christian Wilms
  • Vanessa Closius
  • Ulf Brefeld
  • Simone Frintrop

In this paper, we propose a novel Sign Language Recognition (SLR) model that leverages the task-specific knowledge to incorporate Top-Down (TD) attention to focus the processing of the network on the most relevant parts of the input video sequence. For SLR, this includes information about the hands' shape, orientation and positions, and motion trajectory. Our model consists of three streams that process RGB, optical flow and TD attention data. For the TD attention, we generate pixel-precise attention maps focusing on both hands, thereby retaining valuable hand information, while eliminating distracting background information. Our proposed method outperforms state-of-the-art on a challenging large-scale dataset by over 2%, and achieves strong results with a much simpler architecture compared to other systems on the newly released AUTSL dataset [1].

Original languageEnglish
Title of host publication2023 IEEE International Conference on Image Processing, ICIP 2023 - Proceedings : Proceedings
Number of pages5
Place of PublicationPiscataway
PublisherIEEE Electromagnetic Compatibility Society
Publication date08.10.2023
Pages2555-2559
ISBN (print)978-1-7281-9836-1
ISBN (electronic)978-1-7281-9835-4
DOIs
Publication statusPublished - 08.10.2023
Event2023 IEEE International Conference on Image Processing - Kuala Lumpur Convention Centre, Kuala Lumpur, Malaysia
Duration: 08.10.202311.10.2023
Conference number: 30
https://2023.ieeeicip.org/

Bibliographical note

Publisher Copyright:
© 2023 IEEE.

    Research areas

  • Informatics - sign language recognition, top-down attention, deep learning

Recently viewed

Publications

  1. Recycling of organic residues to produce insulation composites
  2. Simulation of the quench sensitivity of the aluminum alloy 6082
  3. Optimal dynamic scale and structure of a multi-pollution economy
  4. A hypersingular integral equation for the floating body problem
  5. Factorial Validity of the Anxiety Questionnaire for Students (AFS)
  6. Working time preferences and early and late retirement intentions
  7. Anonymized Firm Data under Test: Evidence from a Replication Study
  8. Female Chief Financial Officers (CFOs) and Environmental Decoupling. The moderating impact of Sustainability Board Committees
  9. Income distribution and willingness to pay for ecosystem services
  10. Wirkungen der Beschäftigungspflicht schwerbehinderter Arbeitnehmer
  11. Consistent drivers of plant biodiversity across managed ecosystems
  12. Freie Berufe im Mikrozensus II - Einkommen und Einkommensverteilung
  13. Transfer operator-based extraction of coherent features on surfaces
  14. Offline question answering over linked data using limited resources
  15. Foraging wireworms are attracted to root-produced volatile aldehydes
  16. Machine vision system errors for unmanned aerial vehicle navigation
  17. Employing a Novel Metaheuristic Algorithm to Optimize an LSTM Model
  18. Prospective material flow analysis of the end-of-life decommissioning
  19. Climate change and modelling of extreme temperatures in Switzerland
  20. Humane Orientation as a New Cultural Dimension of the GLOBE Project:
  21. Measurement approaches for inigrated reporting adoption and quality
  22. Detection of oscillations with application in the pantograph control
  23. Intensity of Time and Income Interdependent Multidimensional Poverty:
  24. Abundance of large old trees in wood-pastures of Transylvania (Romania)
  25. Drawing as a Generative Activity and Drawing as a Prognostic Activity
  26. Credit frictions, selection into external finance and gains from trade
  27. Lagrangian analysis of long-term dynamics of turbulent superstructures
  28. The Rise and Fall of Electricity Distribution Cooperatives in Germany
  29. Web-based depression treatment for type 1 and type 2 diabetic patients
  30. Assessing Printability Maps in Additive Manufacturing of Metal Alloys