Principled Interpolation in Normalizing Flows

Research output: Contributions to collected editions/worksArticle in conference proceedingsResearchpeer-review


Generative models based on normalizing flows are very successful in modeling complex data distributions using simpler ones. However, straightforward linear interpolations show unexpected side effects, as interpolation paths lie outside the area where samples are observed. This is caused by the standard choice of Gaussian base distributions and can be seen in the norms of the interpolated samples as they are outside the data manifold. This observation suggests that changing the way of interpolating should generally result in better interpolations, but it is not clear how to do that in an unambiguous way. In this paper, we solve this issue by enforcing a specific manifold and, hence, change the base distribution, to allow for a principled way of interpolation. Specifically, we use the Dirichlet and von Mises-Fisher base distributions on the probability simplex and the hypersphere, respectively. Our experimental results show superior performance in terms of bits per dimension, Frechet Inception Distance (FID), and Kernel Inception Distance (KID) scores for interpolation, while maintaining the generative performance
Original languageEnglish
Title of host publicationMachine Learning and Knowledge Discovery in Databases. Research Track : European Conference, ECML PKDD 2021, Bilbao, Spain, September 13–17, 2021, Proceedings, Part II
EditorsNuria Oliver, Fernando Pérez-Cruz, Stefan Kramer, Jesse Read, Jose A. Lozano
Number of pages16
Place of PublicationCham
PublisherSpringer Nature AG
Publication date09.2021
ISBN (Print)978-3-030-86519-1
ISBN (Electronic)978-3-030-86520-7
Publication statusPublished - 09.2021
EventEuropean Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases - ECML PKDD 2021 - Virtual, Online
Duration: 13.09.202117.09.2021

Bibliographical note

Funding Information:
This research was financed in part by the Coordena??o de Aperfei?oamento de Pessoal de N?vel Superior (CAPES), Brazil, Finance Code 001 and by FAPESP (grants 2017/24005-2 and 2018/19350-5).

Publisher Copyright:
© 2021, Springer Nature Switzerland AG.