We propose a novel generative deep learning architecture based on generative moment matching networks. The objective of our model is to learn a distribution over distributions and generate new sample distributions following the (possibly complex) distribution of training data. We derive a custom loss function for our model based on the maximum mean discrepancy test. Our model is evaluated on different datasets where we investigate the influence of hyperparameters on performance.