Recurrence plots (RPs) have proved to be a very versatile tool to analyze temporal dynamics of time series data. However, it has also been conjectured that RPs can be used to model samples of random variables, that is, data that do not contain any temporal dependencies. In the current paper, we show that RPs can indeed be used to mimic nonparametric inferential statistics. Particularly, we use the case of the two-sample Kolmogorov-Smirnov test as a proof-of-concept, showing how such a test can be done based on RPs. Simulations on differences in mean, variance, and shape of two distributions show that the results of the classical two-sample Kolmogorov-Smirnov test and the recurrence-based test for differences in distributions of two independent samples scale well with each other. While the Kolmogorov-Smirnov test seems to be more sensitive in detecting differences in means, the recurrence based test seems to be more sensitive to detect heteroscedasticity and asymmetry. Potential improvements of our approach as well as extensions to tests with individual distributions are discussed.