Semi-Supervised Generative Models for Multi-Agent Trajectories

Research output: Contributions to collected editions/worksArticle in conference proceedingsResearchpeer-review


Analyzing the spatiotemporal behavior of multiple agents is of great interest to many communities. Existing probabilistic models in this realm are formalized either in an unsupervised framework, where the latent space is described by discrete or continuous variables, or in a supervised framework, where weakly preserved labels add explicit information to continuous latent representations. To overcome inherent limitations, we propose a novel objective function for processing multi-agent trajectories based on semi-supervised variational autoencoders, where equivariance and interaction of agents are captured via customized graph networks. The resulting architecture disentangles discrete and continuous latent effects and provides a natural solution for injecting expensive domain knowledge into interactive sequential systems. Empirically, our model not only outperforms various state-of-the-art baselines in trajectory forecasting, but also learns to effectively leverage unsupervised multi-agent sequences for classification tasks on interactive real-world sports datasets.

Original languageEnglish
Title of host publicationAdvances in Neural Information Processing Systems 35 : 36th Conference on Neural Information Processing Systems (NeurIPS 2022)
EditorsS. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, A. Oh
Number of pages15
Place of PublicationRed Hook
Publication date2022
ISBN (Print)978-1-7138-7108-8
ISBN (Electronic)978-1-7138-7312-9
Publication statusPublished - 2022
Event36th Conference on Neural Information Processing Systems - New Orleans Convention Center, New Orleans, United States
Duration: 28.11.202209.12.2022
Conference number: 36

Bibliographical note

Publisher Copyright:
© 2022 Neural information processing systems foundation. All rights reserved.