PROMPT ENGINEERING FOR REQUIREMENTS ELICITATION: A COMPARATIVE EVALUATION OF EIGHT TECHNIQUES USING O1

Publikation: Beiträge in SammelwerkenAufsätze in KonferenzbändenForschungbegutachtet

Authors

Large language models (LLMs) offer new opportunities for automating requirements elicitation in early-stage product development. However, many existing approaches rely on complex architectures or fine-tuning, which are resource-intensive and often inaccessible to non-experts. This reveals a critical gap: the demand for lightweight, broadly applicable methods that support effective requirements generation without the need for specialized model training. Prompt engineering addresses this gap by guiding pre-trained LLMs through carefully crafted input instructions, thereby avoiding the need for additional training. This study evaluates eight prompt engineering techniques for generating requirements documents using OpenAI’s o1 model on the example of a smart electronic product. The analysis focuses on the structure of requirements lists and examines various methods, ranging from basic approaches (zero-shot, one-shot, and few-shot prompting), to reasoning-based techniques (chain-of-thought and tree-of-thought prompting), and advanced strategies (self-consistency, ensemble refinement, and choice shuffle ensemble). The outputs are assessed based on multiple categories using three LLMs, namely Llama-3.2-3B-Instruct, DeepSeek-R1-Distill-Qwen-7BGGUF, and Mistral-7B-Instruct-v0.3. The results show substantial variation in performance. Simpler prompts often produce incomplete or inconsistent outputs, while more advanced strategies show higher quality and coherence. In particular, combining few-shot prompting with tree-of-thought prompting and ensemble refinement produces the most reliable and well-structured requirements lists. These techniques enable the model to reason more transparently and iteratively refine its outputs. However, challenges remain such as susceptibility to hallucinations and a strong dependence on prompt design.

OriginalspracheEnglisch
Titel51st Design Automation Conference (DAC)
HerausgeberGuang Hao, Dongming Gan, Mark Plecnik, Bingling Huang
Anzahl der Seiten9
VerlagThe American Society of Mechanical Engineers(ASME)
Erscheinungsdatum2025
Aufsatznummerv03bt03a047
ISBN (Print)978-0-7918-8923-7
DOIs
PublikationsstatusErschienen - 2025
VeranstaltungASME 2025 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, IDETC-CIE 2025 - Anaheim, USA / Vereinigte Staaten
Dauer: 17.08.202520.08.2025

Bibliographische Notiz

Publisher Copyright:
Copyright © 2025 by ASME.

DOI