From Feedback to Formative Guidance: Leveraging LLMs for Personalized Support in Programming Projects

Research output: Contributions to collected editions/worksArticle in conference proceedingsResearchpeer-review

Authors

Large Language Models (LLMs) offer scalable opportunities to personalize feedback in education, yet their trustworthiness and effectiveness remain underexplored. We present a study conducted in an introductory programming and data science course with approximately 1,400 first-year university students. A subset of these students received both peer and LLM-generated feedback on their individual programming projects. Our results show that 56% of students preferred the LLM feedback, and 52% could not reliably distinguish it from human-written feedback. Student ratings suggest that LLM feedback is perceived as helpful, constructive, and relevant, though it often lacks personalized depth and motivational nuance. These findings underline the potential of LLMs to support scalable, personalized education, while pointing to key areas for responsible improvement. Based on these insights, we outline the future roadmap for the course in which LLM-generated feedback supports students in their learning journey but also instructors through monitoring student performance and helping to allocate instructional resources more effectively. Given limited human resources this approach enables personalized instructor feedback to be scaled to a large group of students.

Original languageEnglish
Title of host publicationUMAP 2025 - Adjunct Proceedings of the 33rd ACM Conference on User Modeling, Adaptation and Personalization
EditorsCristina Conati, Fedelucio Narducci, Gaetano Rossiello, Cataldo Musto, Julita Vassileva
Number of pages6
PublisherAssociation for Computing Machinery, Inc
Publication date12.06.2025
Pages398-403
ISBN (electronic)9798400713996
DOIs
Publication statusPublished - 12.06.2025
Event33rd Conference on User Modeling, Adaptation and Personalization - UMAP 2025 - New York, United States
Duration: 16.06.202519.06.2025
Conference number: 33

Bibliographical note

Publisher Copyright:
© 2025 Copyright held by the owner/author(s).

    Research areas

  • artificial intelligence, education, large language models, personalized feedback, programming
  • Informatics

DOI