PROMPT ENGINEERING FOR REQUIREMENTS ELICITATION: A COMPARATIVE EVALUATION OF EIGHT TECHNIQUES USING O1
Publikation: Beiträge in Sammelwerken › Aufsätze in Konferenzbänden › Forschung › begutachtet
Authors
Large language models (LLMs) offer new opportunities for automating requirements elicitation in early-stage product development. However, many existing approaches rely on complex architectures or fine-tuning, which are resource-intensive and often inaccessible to non-experts. This reveals a critical gap: the demand for lightweight, broadly applicable methods that support effective requirements generation without the need for specialized model training. Prompt engineering addresses this gap by guiding pre-trained LLMs through carefully crafted input instructions, thereby avoiding the need for additional training. This study evaluates eight prompt engineering techniques for generating requirements documents using OpenAI’s o1 model on the example of a smart electronic product. The analysis focuses on the structure of requirements lists and examines various methods, ranging from basic approaches (zero-shot, one-shot, and few-shot prompting), to reasoning-based techniques (chain-of-thought and tree-of-thought prompting), and advanced strategies (self-consistency, ensemble refinement, and choice shuffle ensemble). The outputs are assessed based on multiple categories using three LLMs, namely Llama-3.2-3B-Instruct, DeepSeek-R1-Distill-Qwen-7BGGUF, and Mistral-7B-Instruct-v0.3. The results show substantial variation in performance. Simpler prompts often produce incomplete or inconsistent outputs, while more advanced strategies show higher quality and coherence. In particular, combining few-shot prompting with tree-of-thought prompting and ensemble refinement produces the most reliable and well-structured requirements lists. These techniques enable the model to reason more transparently and iteratively refine its outputs. However, challenges remain such as susceptibility to hallucinations and a strong dependence on prompt design.
| Originalsprache | Englisch |
|---|---|
| Titel | 51st Design Automation Conference (DAC) |
| Herausgeber | Guang Hao, Dongming Gan, Mark Plecnik, Bingling Huang |
| Anzahl der Seiten | 9 |
| Verlag | The American Society of Mechanical Engineers(ASME) |
| Erscheinungsdatum | 2025 |
| Aufsatznummer | v03bt03a047 |
| ISBN (Print) | 978-0-7918-8923-7 |
| DOIs | |
| Publikationsstatus | Erschienen - 2025 |
| Veranstaltung | ASME 2025 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, IDETC-CIE 2025 - Anaheim, USA / Vereinigte Staaten Dauer: 17.08.2025 → 20.08.2025 |
Bibliographische Notiz
Publisher Copyright:
Copyright © 2025 by ASME.
- Maschinenbau
- Computergrafik und computergestütztes Design
- Angewandte Informatik
- Modellierung und Simulation
ASJC Scopus Sachgebiete
- Ingenieurwissenschaften
