Self-supervised Siamese Autoencoders

Research output: Contributions to collected editions/worksArticle in conference proceedingsResearchpeer-review

Authors

  • Friederike Baier
  • Sebastian Mair
  • Samuel G. Fadel

In contrast to fully-supervised models, self-supervised representation learning only needs a fraction of data to be labeled and often achieves the same or even higher downstream performance. The goal is to pre-train deep neural networks on a self-supervised task, making them able to extract meaningful features from raw input data afterwards. Previously, autoencoders and Siamese networks have been successfully employed as feature extractors for tasks such as image classification. However, both have their individual shortcomings and benefits. In this paper, we combine their complementary strengths by proposing a new method called SidAE (Siamese denoising autoencoder). Using an image classification downstream task, we show that our model outperforms two self-supervised baselines across multiple data sets and scenarios. Crucially, this includes conditions in which only a small amount of labeled data is available. Empirically, the Siamese component has more impact, but the denoising autoencoder is nevertheless necessary to improve performance.

Original languageEnglish
Title of host publicationAdvances in Intelligent Data Analysis XXII : 22nd International Symposium on Intelligent Data Analysis, IDA 2024, Proceedings
EditorsIoanna Miliou, Panagiotis Papapetrou, Nico Piatkowski
Number of pages12
PublisherSpringer Science and Business Media Deutschland
Publication date2024
Pages117-128
ISBN (print)978-3-031-58546-3
ISBN (electronic)978-3-031-58547-0
DOIs
Publication statusPublished - 2024
Event22nd International Symposium on Intelligent Data Analysis - IDA 2024 - NOD building, Stockholm, Stockholm, Sweden
Duration: 24.04.202426.04.2024
Conference number: 22
http://www.wikicfp.com/cfp/servlet/event.showcfp?copyownerid=90704&eventid=176233
http://ida2024.org/

Bibliographical note

Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.

    Research areas

  • denoising autoencoder, image classification, pre-training, representation learning, Self-supervised learning, Siamese networks
  • Informatics
  • Business informatics

Recently viewed

Publications

  1. Utilizing learning analytics to support study success
  2. A dissociation between two classes of spatial abilities in elementary school children
  3. Temporal dynamics of conflict monitoring and the effects of one or two conflict sources on error-(related) negativity
  4. An improved method for the analysis of volatile polyfluorinated alkyl substances in environmental air samples
  5. How people explain their own and others’ behavior:
  6. Added value of convection-permitting simulations for understanding future urban humidity extremes
  7. Survey Response and Observed Behavior
  8. Multi-Professional Support
  9. Absolute and relative maximum strength measures show differences in their correlations with sprint and jump performances in trained youth soccer players
  10. The Change Challenge
  11. Eco product lifecycle management
  12. Development and validation of a questionnaire on parental health literacy in the context of promoting healthy lifestyles during childhood
  13. Stand up for the champion
  14. Aesthetic Practices of the New Right
  15. A path-centric account of action-oriented entrepreneurship training
  16. The rise and decline of regional power
  17. A conceptual cross-disciplinary model of organizational practices for older workers
  18. Creating a Business Case for Sustainability
  19. Multinational Enterprise Strategies for Addressing Sustainability
  20. Introduction
  21. Climate change as an element of sustainability communication
  22. Evaluating a web-based PPGIS for the rehabilitation of urban riparian corridors
  23. Remembering and Communicating Climate Change Narratives – The Influence of World Views on Selective Recollection
  24. Conclusions
  25. Escape. Computerspiele als Kulturtechnik
  26. Interarchive
  27. Digital Workplace Transformation:
  28. Multiply metallated organic intermediates: a tris(lithiomethyl)-cyclohexane and a hexalithiotrimethyl-cyclohexanetriolate.
  29. MännerWeltWald
  30. Asset Management
  31. Who guards the guards with AI-driven robots? The ethicalness and cognitive neutralization of police violence following AI-robot advice