Joint optimization of an autoencoder for clustering and embedding

Research output: Journal contributionsJournal articlesResearchpeer-review

Authors

Deep embedded clustering has become a dominating approach to unsupervised categorization of objects with deep neural networks. The optimization of the most popular methods alternates between the training of a deep autoencoder and a k-means clustering of the autoencoder’s embedding. The diachronic setting, however, prevents the former to benefit from valuable information acquired by the latter. In this paper, we present an alternative where the autoencoder and the clustering are learned simultaneously. This is achieved by providing novel theoretical insight, where we show that the objective function of a certain class of Gaussian mixture models (GMM’s) can naturally be rephrased as the loss function of a one-hidden layer autoencoder thus inheriting the built-in clustering capabilities of the GMM. That simple neural network, referred to as the clustering module, can be integrated into a deep autoencoder resulting in a deep clustering model able to jointly learn a clustering and an embedding. Experiments confirm the equivalence between the clustering module and Gaussian mixture models. Further evaluations affirm the empirical relevance of our deep architecture as it outperforms related baselines on several data sets.

Original languageEnglish
JournalMachine Learning
Volume110
Issue number7
Pages (from-to)1901-1937
Number of pages37
ISSN0885-6125
DOIs
Publication statusPublished - 01.07.2021

Documents

DOI

Recently viewed

Activities

  1. Domestication and/or Digital Divide – How to Overcome Binary Classifications in Analysing Everyday Internet Use and Diffusion
  2. Optimal trajectory generation using MPC in robotino and its implementation with ROS system
  3. Temporary Organizing and Organizing Trmporality: On the Multilayered Architecture of Accelerators
  4. The Domestication Approach Revisited in the Context of Digitization, Mobilization and Mediatization
  5. Using a Longitudinal Mixed-Methods Approach in HESD Research: Reflections on Pitfalls and Added Value
  6. A Learning Agent for Parameter Adaptation in Speeded Tests
  7. Lagrangian aspects of turbulent superstructures: numerical analysis of long-term dynamics and transport properties
  8. How stereotypes affect grading and tutorial feedback: Shifting evaluations or shifting standards?
  9. The importance of scales for legitimate and effective participatory environmental governance. Findings from a multi-level comparative case study of implementing the Water Framework Directive (with D. Schulz)
  10. Can better texts support weak students? Interactions between text features and readers' abilities
  11. International Conference on Methods and Models in Automation an Robotics - MMAR 2016
  12. Capitalizing on value dynamics
  13. The golden age of software architecture better named the middle age of software architecture - Some provocative thoughts
  14. Creating pathways for transformation through amplifying approaches: a case-study from Southern Transylvania
  15. E-learning module on “Participation” in the context of IWRM – “Social Science” Part

Publications

  1. Managing Business Process in Distributed Systems: Requirements, Models, and Implementation
  2. Using Natural Language Processing Techniques to Tackle the Construct Identity Problem in Information Systems Research
  3. Modeling Effective and Ineffective Knowledge Communication and Learning Discourses in CSCL with Hidden Markov Models
  4. Knowledge Graph Question Answering Using Graph-Pattern Isomorphism
  5. Evaluation of Time/Phase Parameters in Frequency Measurements for Inertial Navigation Systems
  6. A Multilevel CFA-MTMM Model for Nested Structurally Different Methods
  7. Dynamic Lot Size Optimization with Reinforcement Learning
  8. Parking space management through deep learning – an approach for automated, low-cost and scalable real-time detection of parking space occupancy
  9. Latent structure perceptron with feature induction for unrestricted coreference resolution
  10. Effectiveness of a guided multicomponent internet and mobile gratitude training program - A pragmatic randomized controlled trial
  11. The signal location task as a method quantifying the distribution of attention
  12. Understanding the properties of isospectral points and pairs in graphs
  13. A Review of Latent Variable Modeling Using R - A Step-by-Step-Guide
  14. Mirrored piezo servo hydraulic actuators for use in camless combustion engines and its Control with mirrored inputs and MPC
  15. Spaces for challenging experiences, indeterminacy, and experimentation
  16. Sliding-Mode-Based Input-Output Linearization of a Peltier Element for Ice Clamping Using a State and Disturbance Observer
  17. Using Heider’s Epistemology of Thing and Medium for Unpacking the Conception of Documents: Gantt Charts and Boundary Objects
  18. Need Satisfaction and Optimal Functioning at Leisure and Work: A Longitudinal Validation Study of the DRAMMA Model
  19. Modelling, Simulation and Experimental Analysis of a Metal-Polymer Hybrid Fibre based Microstrip Resonator for High Frequency Characterisation
  20. Control versus Complexity
  21. New method for assessing the repeatability of the measuring system for roughness measurements
  22. How to support synchronous net-based learning discourses
  23. Collaborative open science as a way to reproducibility and new insights in primate cognition research
  24. Guest Editorial - ''Econometrics of Anonymized Micro Data''
  25. “Ideation is Fine, but Execution is Key”
  26. Incorporating ecosystem services into ecosystem-based management to deal with complexity
  27. Deconstructing the Theoretical Language of Process Research
  28. Computational modeling of amorphous polymers