Joint optimization of an autoencoder for clustering and embedding

Publikation: Beiträge in ZeitschriftenZeitschriftenaufsätzeForschungbegutachtet

Authors

Deep embedded clustering has become a dominating approach to unsupervised categorization of objects with deep neural networks. The optimization of the most popular methods alternates between the training of a deep autoencoder and a k-means clustering of the autoencoder’s embedding. The diachronic setting, however, prevents the former to benefit from valuable information acquired by the latter. In this paper, we present an alternative where the autoencoder and the clustering are learned simultaneously. This is achieved by providing novel theoretical insight, where we show that the objective function of a certain class of Gaussian mixture models (GMM’s) can naturally be rephrased as the loss function of a one-hidden layer autoencoder thus inheriting the built-in clustering capabilities of the GMM. That simple neural network, referred to as the clustering module, can be integrated into a deep autoencoder resulting in a deep clustering model able to jointly learn a clustering and an embedding. Experiments confirm the equivalence between the clustering module and Gaussian mixture models. Further evaluations affirm the empirical relevance of our deep architecture as it outperforms related baselines on several data sets.

OriginalspracheEnglisch
ZeitschriftMachine Learning
Jahrgang110
Ausgabenummer7
Seiten (von - bis)1901-1937
Anzahl der Seiten37
ISSN0885-6125
DOIs
PublikationsstatusErschienen - 01.07.2021

Dokumente

DOI

Zuletzt angesehen

Publikationen

  1. Median based algorithm as an entropy function for noise detectionin wavelet trees for data reconciliation
  2. Prediction of the tool change point in a polishing process using a modular software framework
  3. Preventive Emergency Detection Based on the Probabilistic Evaluation of Distributed, Embedded Sensor Networks
  4. Heuristic approximation and computational algorithms for closed networks
  5. Parsing Causal Models – An Instance Segmentation Approach
  6. Using haar wavelets for fault detection in technical processes
  7. Detection and mapping of water pollution variation in the Nile Delta using multivariate clustering and GIS techniques
  8. Computational modeling of material flow networks
  9. Inversion of Fuzzy Neural Networks for the Reduction of Noise in the Control Loop for Automotive Applications
  10. Wavelet based Fault Detection and RLS Parameter Estimation of Conductive Fibers with a Simultaneous Estimation of Time-Varying Disturbance
  11. ACL–adaptive correction of learning parameters for backpropagation based algorithms
  12. Finding Similar Movements in Positional Data Streams
  13. Learning Rotation Sensitive Neural Network for Deformed Objects' Detection in Fisheye Images
  14. A two-step approach for the prediction of mood levels based on diary data
  15. Modeling and Performance Analysis of a Node in Fault Tolerant Wireless Sensor Networks
  16. Evaluating OWL 2 reasoners in the context of checking entity-relationship diagrams during software development
  17. Using trait-based filtering as a predictive framework for conservation
  18. A Multivariate Method for Dynamic System Analysis
  19. Authenticity and authentication in language learning
  20. Supervised clustering of streaming data for email batch detection
  21. Modified dynamic programming approach for offline segmentation of long hydrometeorological time series
  22. A geometric algorithm for the output functional controllability in general manipulation systems and mechanisms
  23. Analysis of Complexity Reduction in Kalman Filters Through Decoupling Control With Chattered Inputs in PMSM
  24. Substructure, subgraph, and walk counts as measures of the complexity of graphs and molecules.
  25. Modeling precipitation kinetics for multi-phase and multi-component systems using particle size distributions via a moving grid technique
  26. Homogenization modeling of thin-layer-type microstructures
  27. Multi-view learning with dependent views
  28. Machine Learning and Knowledge Discovery in Databases
  29. Model inversion using fuzzy neural network with boosting of the solution
  30. Using Complexity Metrics to Assess Silent Reading Fluency
  31. Comparing the Sensitivity of Social Networks, Web Graphs, and Random Graphs with Respect to Vertex Removal