PROMPT ENGINEERING FOR REQUIREMENTS ELICITATION: A COMPARATIVE EVALUATION OF EIGHT TECHNIQUES USING O1
Research output: Contributions to collected editions/works › Article in conference proceedings › Research › peer-review
Authors
Large language models (LLMs) offer new opportunities for automating requirements elicitation in early-stage product development. However, many existing approaches rely on complex architectures or fine-tuning, which are resource-intensive and often inaccessible to non-experts. This reveals a critical gap: the demand for lightweight, broadly applicable methods that support effective requirements generation without the need for specialized model training. Prompt engineering addresses this gap by guiding pre-trained LLMs through carefully crafted input instructions, thereby avoiding the need for additional training. This study evaluates eight prompt engineering techniques for generating requirements documents using OpenAI’s o1 model on the example of a smart electronic product. The analysis focuses on the structure of requirements lists and examines various methods, ranging from basic approaches (zero-shot, one-shot, and few-shot prompting), to reasoning-based techniques (chain-of-thought and tree-of-thought prompting), and advanced strategies (self-consistency, ensemble refinement, and choice shuffle ensemble). The outputs are assessed based on multiple categories using three LLMs, namely Llama-3.2-3B-Instruct, DeepSeek-R1-Distill-Qwen-7BGGUF, and Mistral-7B-Instruct-v0.3. The results show substantial variation in performance. Simpler prompts often produce incomplete or inconsistent outputs, while more advanced strategies show higher quality and coherence. In particular, combining few-shot prompting with tree-of-thought prompting and ensemble refinement produces the most reliable and well-structured requirements lists. These techniques enable the model to reason more transparently and iteratively refine its outputs. However, challenges remain such as susceptibility to hallucinations and a strong dependence on prompt design.
| Original language | English |
|---|---|
| Title of host publication | 51st Design Automation Conference (DAC) |
| Editors | Guang Hao, Dongming Gan, Mark Plecnik, Bingling Huang |
| Number of pages | 9 |
| Publisher | The American Society of Mechanical Engineers(ASME) |
| Publication date | 2025 |
| Article number | v03bt03a047 |
| ISBN (print) | 978-0-7918-8923-7 |
| DOIs | |
| Publication status | Published - 2025 |
| Event | ASME 2025 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, IDETC-CIE 2025 - Anaheim, United States Duration: 17.08.2025 → 20.08.2025 |
Bibliographical note
Publisher Copyright:
Copyright © 2025 by ASME.
- Mechanical Engineering
- Computer Graphics and Computer-Aided Design
- Computer Science Applications
- Modelling and Simulation
ASJC Scopus Subject Areas
- Large Language Models, Product Development, Prompt Engineering, Requirements Elicitation
- Engineering
