Self-supervised Siamese Autoencoders

Research output: Contributions to collected editions/worksArticle in conference proceedingsResearchpeer-review

Authors

  • Friederike Baier
  • Sebastian Mair
  • Samuel G. Fadel

In contrast to fully-supervised models, self-supervised representation learning only needs a fraction of data to be labeled and often achieves the same or even higher downstream performance. The goal is to pre-train deep neural networks on a self-supervised task, making them able to extract meaningful features from raw input data afterwards. Previously, autoencoders and Siamese networks have been successfully employed as feature extractors for tasks such as image classification. However, both have their individual shortcomings and benefits. In this paper, we combine their complementary strengths by proposing a new method called SidAE (Siamese denoising autoencoder). Using an image classification downstream task, we show that our model outperforms two self-supervised baselines across multiple data sets and scenarios. Crucially, this includes conditions in which only a small amount of labeled data is available. Empirically, the Siamese component has more impact, but the denoising autoencoder is nevertheless necessary to improve performance.

Original languageEnglish
Title of host publicationAdvances in Intelligent Data Analysis XXII : 22nd International Symposium on Intelligent Data Analysis, IDA 2024, Proceedings
EditorsIoanna Miliou, Panagiotis Papapetrou, Nico Piatkowski
Number of pages12
PublisherSpringer Science and Business Media Deutschland
Publication date2024
Pages117-128
ISBN (print)978-3-031-58546-3
ISBN (electronic)978-3-031-58547-0
DOIs
Publication statusPublished - 2024
Event22nd International Symposium on Intelligent Data Analysis - IDA 2024 - NOD building, Stockholm, Stockholm, Sweden
Duration: 24.04.202426.04.2024
Conference number: 22
http://www.wikicfp.com/cfp/servlet/event.showcfp?copyownerid=90704&eventid=176233
http://ida2024.org/

Bibliographical note

Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.

    Research areas

  • denoising autoencoder, image classification, pre-training, representation learning, Self-supervised learning, Siamese networks
  • Informatics
  • Business informatics

Recently viewed

Publications

  1. Tree cover mediates the effect on rapeseed leaf damage of excluding predatory arthropods, but in an unexpected way
  2. Mapping Complexity in Environmental Governance
  3. Comparing Instrument-induced effects in EFL requests
  4. Detection of significant tracer gases by means of polymer gas sensors
  5. Is Calluna vulgaris a suitable bio-monitor of management-mediated nutrient pools in heathland ecosystems?
  6. Scaffolding, software and scenarios
  7. Bordering the Area of Spatial Relevance for Schools
  8. Common Ground and Development
  9. Towards a Model for Building Trust and Acceptance of Artificial Intelligence Aided Medical Assessment Systems
  10. Optimal grazing management rules in semi-arid rangelands with uncertain rainfall
  11. A sensitive microsystem as biosensor for cell growth monitoring and antibiotic testing
  12. The effectiveness of nudging
  13. Influences of yttrium content on microstructure and mechanical properties of as-cast Mg–Ca–Y–Zr alloys
  14. Absolute and relative maximum strength measures show differences in their correlations with sprint and jump performances in trained youth soccer players
  15. DECODING SUSTAINABILITY IN THE HEALTHCARE SYSTEM. TEACHING STUDENTS HOW TO PROBLEMATIZE COMPLEX CONCEPTS
  16. The challenges of gamifying CSR communication
  17. Preference and willingness to pay for meat substitutes based on micro-algae
  18. Improved mechanical properties of cast Mg alloy welds via texture weakening by differential rotation refill friction stir spot welding
  19. The patterns of curriculum change processes that embed sustainability in higher education institutions
  20. Simultaneous Determination of 11 Sulfonamides by HPLC–UV and Application for Fast Screening of Their Aerobic Elimination and Biodegradation in a Simple Test
  21. The Role of Public Participation in Managing Uncertainty in the Implementation of the Water Framework Directive
  22. Determiner Ellipsis in Electronic Writing - Discourse or Syntax?
  23. Gehen in der Datenbank – Der BMLwalker
  24. Sustainability Transitions and the Spatial Interface: Developing Conceptual Perspectives
  25. Replik
  26. Organizing Colour
  27. Evaluation of a temporal causal model for predicting the mood of clients in an online therapy
  28. Almost-invariant sets and invariant manifolds
  29. Knowledge and social learning for sustainable development
  30. Norms and variation in L2 pragmatics
  31. Does Social Exclusion Improve Detection of Real and Fake Smiles?
  32. Medial erzeugte Befindlichkeiten
  33. Lifelong learning in practice at Leuphana University
  34. Productivity and size of the export market
  35. Repräsentative Wahlstatistik
  36. Using a leverage points perspective to compare social-ecological systems: a case study on rural landscapes
  37. Greater fit and a greater gap
  38. Case Study: Between radiation and imagination
  39. Formation mechanism of the abnormal texture during extrusion in Mg-Y-Sm-Zn-Zr alloy
  40. Article 6
  41. Continous preventive diagnosis for cardiovascular diseases based on stochastic modeling
  42. Anisotropic wavelet bases and thresholding
  43. AAL-Onto