In industrial deep drawing processes, stochastic fluctuations and disturbances of the manufacturing conditions occur, which can cause uncontrolled deterioration of the product properties. The immunity to these negative influences is referred to as robustness. Robustness in deep drawing can be assessed by sensors integrated into the press line. This generates extensive amounts of data that have potential to be used for machine learning modelling and for analysing complex interactions. The field of explainable AI, which serves to explain such data-driven models is becoming increasingly relevant.