The field of machine learning (ML) is of specific interest for production companies as it displays a perspective to handle the increased complexity within their production planning and control (PPC) processes in an economic and ecologic effective as well as efficient way. Several studies investigate applications of ML to different use cases. However, the research field lacks in research on industry case studies. A broad understanding from a practical perspective and in this context, an evaluation from a data mining and business standpoint is key for gaining trust in ML solutions. Therefore, this paper gives a comprehensive overview of evaluation dimensions and outlines the current state of research in ML-PPC by conducting a systematic research overview. First, the present work provides key dimensions of business and data mining objectives as evaluation metric. Business objectives are clustered into economic, ecological and social objectives and data mining objectives are grouped into prediction accuracy, model’s explainability, model’s runtime, and model’s energy use. Secondly, the systematic literature review identifies 45 industry case studies in ML-PPC from 2010-2020. The work shows that the scientific publications only rarely reflect in detail on a wide range of evaluation metrics. Instead, researchers mainly focus on prediction accuracy and seldom investigate the effect of their results to a business context. Positively, some papers reflect on further aspects and can inspire future research. This resulting transparency supports decision makers of companies in their prioritization process when setting up a future ML-roadmap. In addition, the research gaps identified herein invite researchers to join the research field.