Industrial sheet metal components often have complex geometries with both concave and convex features. For small batch sizes, such components can be manufactured by incremental sheet forming, using the pressure of an active medium underneath the workpiece to create the convex feature. However, the additional load superimposed by the pressure causes instability and renders the process more prone to failure, in particular to cracking of the workpiece. The reliability of the manufacturing process could be improved if the occurrence of failure were predictable and thus preventable. To achieve this goal, the trend of the forming forces and the change of the workpiece geometry prior to cracking are experimentally analyzed. Subsequently, the dataset obtained from the experiments is used to fit a model based on long short-term memory and on a sliding window approach. This model reliably predicts the probability of failure with an accuracy and recall of 0.97 and 0.89 respectively, demonstrating its potential for online monitoring of the manufacturing process.