Self-supervised Siamese Autoencoders

Research output: Contributions to collected editions/worksArticle in conference proceedingsResearchpeer-review

Authors

  • Friederike Baier
  • Sebastian Mair
  • Samuel G. Fadel

In contrast to fully-supervised models, self-supervised representation learning only needs a fraction of data to be labeled and often achieves the same or even higher downstream performance. The goal is to pre-train deep neural networks on a self-supervised task, making them able to extract meaningful features from raw input data afterwards. Previously, autoencoders and Siamese networks have been successfully employed as feature extractors for tasks such as image classification. However, both have their individual shortcomings and benefits. In this paper, we combine their complementary strengths by proposing a new method called SidAE (Siamese denoising autoencoder). Using an image classification downstream task, we show that our model outperforms two self-supervised baselines across multiple data sets and scenarios. Crucially, this includes conditions in which only a small amount of labeled data is available. Empirically, the Siamese component has more impact, but the denoising autoencoder is nevertheless necessary to improve performance.

Original languageEnglish
Title of host publicationAdvances in Intelligent Data Analysis XXII : 22nd International Symposium on Intelligent Data Analysis, IDA 2024, Proceedings
EditorsIoanna Miliou, Panagiotis Papapetrou, Nico Piatkowski
Number of pages12
PublisherSpringer Science and Business Media Deutschland
Publication date2024
Pages117-128
ISBN (print)978-3-031-58546-3
ISBN (electronic)978-3-031-58547-0
DOIs
Publication statusPublished - 2024
Event22nd International Symposium on Intelligent Data Analysis - IDA 2024 - NOD building, Stockholm, Stockholm, Sweden
Duration: 24.04.202426.04.2024
Conference number: 22
http://www.wikicfp.com/cfp/servlet/event.showcfp?copyownerid=90704&eventid=176233
http://ida2024.org/

Bibliographical note

Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.

    Research areas

  • denoising autoencoder, image classification, pre-training, representation learning, Self-supervised learning, Siamese networks
  • Informatics
  • Business informatics

Recently viewed

Publications

  1. Noise level estimation and detection
  2. Mathematics in Robot Control for Theoretical and Applied Problems
  3. Wavelet functions for rejecting spurious values
  4. Applying the Three Horizons approach in local and regional scenarios to support policy coherence in SDG implementation
  5. The use of functional traits in assessing productivity in natural ecosystems
  6. Machine Learning For Determining Planned Order Lead Times In Job Shop Production: A Systematic Review Of Input Factors And Applied Methods
  7. Mining Implications From Data
  8. Factor structure and measurement invariance of the Students’ Self-report Checklist of Social and Learning Behaviour (SSL)
  9. Modernizing persistence–bioaccumulation–toxicity (PBT) assessment with high throughput animal-free methods
  10. Control oriented modeling of DCDC converters
  11. Exploring large vegetation databases to detect temporal trends in species occurrences
  12. Toward Data-Driven Analyses of Electronic Text Books
  13. Some results on output algebraic feedback with applications to mechanical systems
  14. Service Level Driven Stock Allocation
  15. Internet-based public debate of CCS
  16. The Framework for Inclusive Science Education
  17. Finite element based determination and optimization of seam weld positions in porthole die extrusion of double hollow profile with asymmetric cross section
  18. Inventing a secure future
  19. Modeling High Aswan Dam Reservoir Morphology Using Remote Sensing to Reduce Evaporation
  20. Open Innovation in Schools
  21. Kinetic modeling of active plasma resonance spectroscopy
  22. Disabling barriers—Coping with accessibility of nature in Biosphere Reserves
  23. Grassroots social innovation and the mobilisation of values in collaborative consumption
  24. Increasing personal initiative in small business managers or owners leads to entrepreneurial success: A theory-based controlled randomized field intervention for evidence-based management
  25. Propagating Maximum Capacities for Recommendation
  26. The multipole resonance probe
  27. The impact of (social) anchors on Prospect Theory's value function
  28. Change and Continuity in Western Welfare Practices
  29. 'Crawling Jurisdiction'
  30. Global decoupling of functional and phylogenetic diversity in plant communities
  31. Not invented here, not shared here: How school leaders’ attitudes towards external knowledge affect collaborative innovation and collective teacher innovativeness in Germany
  32. Block matrix based LU decomposition to analyze kinetic damping in active plasma resonance spectroscopy