The importance of sport enjoyment for a long-term commitment to physical activity is well established. Thus, enjoyment in doing sports should ideally be promoted in physical education at school. The aim of the present study was to assess the influence of seven factors (perceived competence, social relatedness, social interaction, autonomy, teaching skills, general physical ability, parental support) on enjoyment in physical education. In all, 1,598 pupils from grades 7–10 took part in this study. Sport enjoyment was measured with three scales (pleasure, flow experience, recovery). The results indicate moderate-to-strong correlations of the three facets of sport enjoyment with all factors. A strong positive influence of perceived competence and social relatedness on the experience of sport enjoyment in physical education was found. Teaching skills had a small effect, and for parental support an indirect effect via general physical ability was found. These results offer promising evidence for designing interventions for the facilitation of sport enjoyment in physical education classes.