Self-supervised Siamese Autoencoders

Research output: Contributions to collected editions/worksArticle in conference proceedingsResearchpeer-review

Authors

  • Friederike Baier
  • Sebastian Mair
  • Samuel G. Fadel

In contrast to fully-supervised models, self-supervised representation learning only needs a fraction of data to be labeled and often achieves the same or even higher downstream performance. The goal is to pre-train deep neural networks on a self-supervised task, making them able to extract meaningful features from raw input data afterwards. Previously, autoencoders and Siamese networks have been successfully employed as feature extractors for tasks such as image classification. However, both have their individual shortcomings and benefits. In this paper, we combine their complementary strengths by proposing a new method called SidAE (Siamese denoising autoencoder). Using an image classification downstream task, we show that our model outperforms two self-supervised baselines across multiple data sets and scenarios. Crucially, this includes conditions in which only a small amount of labeled data is available. Empirically, the Siamese component has more impact, but the denoising autoencoder is nevertheless necessary to improve performance.

Original languageEnglish
Title of host publicationAdvances in Intelligent Data Analysis XXII : 22nd International Symposium on Intelligent Data Analysis, IDA 2024, Proceedings
EditorsIoanna Miliou, Panagiotis Papapetrou, Nico Piatkowski
Number of pages12
PublisherSpringer Science and Business Media Deutschland GmbH
Publication date2024
Pages117-128
ISBN (print)978-3-031-58546-3
ISBN (electronic)978-3-031-58547-0
DOIs
Publication statusPublished - 2024
Event22nd International Symposium on Intelligent Data Analysis - IDA 2024 - NOD building, Stockholm, Stockholm, Sweden
Duration: 24.04.202426.04.2024
Conference number: 22
http://www.wikicfp.com/cfp/servlet/event.showcfp?copyownerid=90704&eventid=176233
http://ida2024.org/

Bibliographical note

Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.

    Research areas

  • denoising autoencoder, image classification, pre-training, representation learning, Self-supervised learning, Siamese networks
  • Informatics
  • Business informatics

Recently viewed

Publications

  1. Early Edema Detection Based on the Examination of Multidimensional Ultra-Wide band Data
  2. Offline question answering over linked data using limited resources
  3. Validation of Inspection Frameworks and Methods
  4. Dynamic control of internal force for visco-elastic contact grasps
  5. Multiscale solutions of the electromagnetic continuity differential equation using packets of harmonic wavelets
  6. Optical 3D laser measurement system for navigation of autonomous mobile robot
  7. An introduction to sliding mode control for interdisciplinary education
  8. Predicate‐based model of problem‐solving for robotic actions planning
  9. A toolkit for robust risk assessment using F-divergences
  10. On the Equivalence of Transmission Problems in Nonoverlapping Domain Decomposition Methods for Quasilinear PDEs
  11. A Trajectory Generation Algorithm for Optimal Consumption in Electromagnetic Actuators
  12. Switching cascade controllers combined with a feedforward regulation for an aggregate actuator in automotive applications
  13. Design of Reliable Remobilisation Finger Implants with Geometry Elements of a Triple Periodic Minimal Surface Structure via Additive Manufacturing of Silicon Nitride
  14. On the role of linguistic features for comprehension and learning from STEM texts. A meta-analysis
  15. Elemental analysis using electron beam‐induced K X‐rays
  16. Sliding Mode Control Strategies for Maglev Systems Based on Kalman Filtering
  17. Self-perceived quality of life predicts mortality risk better than a multi-biomarker panel, but the combination of both does best
  18. Multiobjective optimal control of fluid mixing
  19. Trajectory tracking using MPC and a velocity observer for flat actuator systems in automotive applications
  20. Toward Data-Driven Analyses of Electronic Text Books
  21. Design, Modeling and Control of an Over-actuated Hexacopter Tilt-Rotor
  22. Distributable Modular Software Framework for Manufacturing Systems
  23. Confidence levels and likelihood terms in IPCC reports
  24. Determination of the construction and the material identity values of outside building components with the help of in-situ measuring procedures and FEM-simulation calculations
  25. Natural enemy diversity reduces temporal variability in wasp but not bee parasitism
  26. Microstructure and mechanical properties of as-cast Mg-Sn-Ca alloys and effect of alloying elements
  27. Continued logarithm representation of real numbers
  28. Graph-based Approaches for Analyzing Team Interaction on the Example of Soccer
  29. Principals between exploitation and exploration
  30. Geometric series with randomly increasing exponents
  31. Do Linguistic Features Influence Item Difficulty in Physics Assessments?
  32. Detection of oscillations with application in the pantograph control
  33. A dissociation between two classes of spatial abilities in elementary school children
  34. Export Boosting Policies and Firm Performance
  35. Exploring the implications of the value concept for performance assessment of sustainable business models
  36. A piezo servo hydraulic actuator for use in camless combustion engines and its control with MPC
  37. Smart Multi-coil Inductive Power Tranmission with IoT Based Visulization
  38. Light availability and land-use history drive biodiversity and functional changes in forest herb layer communities
  39. Chronic effects of a static stretching intervention program on range of motion and tissue hardness in older adults
  40. Transdisciplinary co-creation increases the utilization of knowledge from sustainable development research
  41. A Sensitive Microsystem as Biosensor for Cell Growth Monitoring and Antibiotic Testing
  42. Logistical Potentials of Load Balancing via the Build-up and Reduction of Stock
  43. Discrete Lyapunov Controllers for an Actuator in Camless Engines
  44. Control of a two-thermoelectric-cooler system for ice-clamping application using Lyapunov based approach
  45. Knowledge Graph Question Answering Datasets and Their Generalizability
  46. An intersection test for the cointegrating rank in dependent panel data
  47. Improve a 3D distance measurement accuracy in stereo vision systems using optimization methods’ approach