Principled Interpolation in Normalizing Flows

Research output: Contributions to collected editions/worksArticle in conference proceedingsResearchpeer-review

Standard

Principled Interpolation in Normalizing Flows. / Fadel, Samuel; Mair, Sebastian; da Silva Torres, Ricardo et al.
Machine Learning and Knowledge Discovery in Databases. Research Track: European Conference, ECML PKDD 2021, Bilbao, Spain, September 13–17, 2021, Proceedings, Part II. ed. / Nuria Oliver; Fernando Pérez-Cruz; Stefan Kramer; Jesse Read; Jose A. Lozano. Cham: Springer Nature AG, 2021. p. 116-131 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12976 LNAI).

Research output: Contributions to collected editions/worksArticle in conference proceedingsResearchpeer-review

Harvard

Fadel, S, Mair, S, da Silva Torres, R & Brefeld, U 2021, Principled Interpolation in Normalizing Flows. in N Oliver, F Pérez-Cruz, S Kramer, J Read & JA Lozano (eds), Machine Learning and Knowledge Discovery in Databases. Research Track: European Conference, ECML PKDD 2021, Bilbao, Spain, September 13–17, 2021, Proceedings, Part II. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12976 LNAI, Springer Nature AG, Cham, pp. 116-131, European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases - ECML PKDD 2021, Virtual, Online, 13.09.21. https://doi.org/10.1007/978-3-030-86520-7_8

APA

Fadel, S., Mair, S., da Silva Torres, R., & Brefeld, U. (2021). Principled Interpolation in Normalizing Flows. In N. Oliver, F. Pérez-Cruz, S. Kramer, J. Read, & J. A. Lozano (Eds.), Machine Learning and Knowledge Discovery in Databases. Research Track: European Conference, ECML PKDD 2021, Bilbao, Spain, September 13–17, 2021, Proceedings, Part II (pp. 116-131). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12976 LNAI). Springer Nature AG. https://doi.org/10.1007/978-3-030-86520-7_8

Vancouver

Fadel S, Mair S, da Silva Torres R, Brefeld U. Principled Interpolation in Normalizing Flows. In Oliver N, Pérez-Cruz F, Kramer S, Read J, Lozano JA, editors, Machine Learning and Knowledge Discovery in Databases. Research Track: European Conference, ECML PKDD 2021, Bilbao, Spain, September 13–17, 2021, Proceedings, Part II. Cham: Springer Nature AG. 2021. p. 116-131. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). Epub 2021. doi: 10.1007/978-3-030-86520-7_8

Bibtex

@inbook{3266e4f6f901427a84649ba43db76f8f,
title = "Principled Interpolation in Normalizing Flows",
abstract = "Generative models based on normalizing flows are very successful in modeling complex data distributions using simpler ones. However, straightforward linear interpolations show unexpected side effects, as interpolation paths lie outside the area where samples are observed. This is caused by the standard choice of Gaussian base distributions and can be seen in the norms of the interpolated samples as they are outside the data manifold. This observation suggests that changing the way of interpolating should generally result in better interpolations, but it is not clear how to do that in an unambiguous way. In this paper, we solve this issue by enforcing a specific manifold and, hence, change the base distribution, to allow for a principled way of interpolation. Specifically, we use the Dirichlet and von Mises-Fisher base distributions on the probability simplex and the hypersphere, respectively. Our experimental results show superior performance in terms of bits per dimension, Frechet Inception Distance (FID), and Kernel Inception Distance (KID) scores for interpolation, while maintaining the generative performance",
keywords = "Informatics, Generative Modeling, Density estimation, Normalizing Flows, Business informatics",
author = "Samuel Fadel and Sebastian Mair and {da Silva Torres}, Ricardo and Ulf Brefeld",
note = "Funding Information: This research was financed in part by the Coordena??o de Aperfei?oamento de Pessoal de N?vel Superior (CAPES), Brazil, Finance Code 001 and by FAPESP (grants 2017/24005-2 and 2018/19350-5). Publisher Copyright: {\textcopyright} 2021, Springer Nature Switzerland AG.; European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases - ECML PKDD 2021, ECML PKDD ; Conference date: 13-09-2021 Through 17-09-2021",
year = "2021",
month = sep,
doi = "10.1007/978-3-030-86520-7_8",
language = "English",
isbn = "978-3-030-86519-1",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Nature AG",
pages = "116--131",
editor = "Nuria Oliver and Fernando P{\'e}rez-Cruz and Stefan Kramer and Jesse Read and Lozano, {Jose A.}",
booktitle = "Machine Learning and Knowledge Discovery in Databases. Research Track",
address = "Germany",

}

RIS

TY - CHAP

T1 - Principled Interpolation in Normalizing Flows

AU - Fadel, Samuel

AU - Mair, Sebastian

AU - da Silva Torres, Ricardo

AU - Brefeld, Ulf

N1 - Funding Information: This research was financed in part by the Coordena??o de Aperfei?oamento de Pessoal de N?vel Superior (CAPES), Brazil, Finance Code 001 and by FAPESP (grants 2017/24005-2 and 2018/19350-5). Publisher Copyright: © 2021, Springer Nature Switzerland AG.

PY - 2021/9

Y1 - 2021/9

N2 - Generative models based on normalizing flows are very successful in modeling complex data distributions using simpler ones. However, straightforward linear interpolations show unexpected side effects, as interpolation paths lie outside the area where samples are observed. This is caused by the standard choice of Gaussian base distributions and can be seen in the norms of the interpolated samples as they are outside the data manifold. This observation suggests that changing the way of interpolating should generally result in better interpolations, but it is not clear how to do that in an unambiguous way. In this paper, we solve this issue by enforcing a specific manifold and, hence, change the base distribution, to allow for a principled way of interpolation. Specifically, we use the Dirichlet and von Mises-Fisher base distributions on the probability simplex and the hypersphere, respectively. Our experimental results show superior performance in terms of bits per dimension, Frechet Inception Distance (FID), and Kernel Inception Distance (KID) scores for interpolation, while maintaining the generative performance

AB - Generative models based on normalizing flows are very successful in modeling complex data distributions using simpler ones. However, straightforward linear interpolations show unexpected side effects, as interpolation paths lie outside the area where samples are observed. This is caused by the standard choice of Gaussian base distributions and can be seen in the norms of the interpolated samples as they are outside the data manifold. This observation suggests that changing the way of interpolating should generally result in better interpolations, but it is not clear how to do that in an unambiguous way. In this paper, we solve this issue by enforcing a specific manifold and, hence, change the base distribution, to allow for a principled way of interpolation. Specifically, we use the Dirichlet and von Mises-Fisher base distributions on the probability simplex and the hypersphere, respectively. Our experimental results show superior performance in terms of bits per dimension, Frechet Inception Distance (FID), and Kernel Inception Distance (KID) scores for interpolation, while maintaining the generative performance

KW - Informatics

KW - Generative Modeling

KW - Density estimation

KW - Normalizing Flows

KW - Business informatics

UR - http://www.scopus.com/inward/record.url?scp=85115734547&partnerID=8YFLogxK

UR - https://www.mendeley.com/catalogue/8bb702c5-63b3-3bd3-a647-8605a1e2eca2/

U2 - 10.1007/978-3-030-86520-7_8

DO - 10.1007/978-3-030-86520-7_8

M3 - Article in conference proceedings

AN - SCOPUS:85115734547

SN - 978-3-030-86519-1

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 116

EP - 131

BT - Machine Learning and Knowledge Discovery in Databases. Research Track

A2 - Oliver, Nuria

A2 - Pérez-Cruz, Fernando

A2 - Kramer, Stefan

A2 - Read, Jesse

A2 - Lozano, Jose A.

PB - Springer Nature AG

CY - Cham

T2 - European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases - ECML PKDD 2021

Y2 - 13 September 2021 through 17 September 2021

ER -