Improving LLM-assisted Requirement Validation Using Persona-based Prompting

Jati Hiliamsyah Husen; Mira Kania Sabariah; Daffa Hilmy Fadhlurohman; Adam Rafif Faqih

doi:10.33365/jatika.v6i3.496

Jati H. Husen School of Computing, Telkom University
Mira Kania Sabariah School of Computing, Telkom University
Daffa Hilmy Fadhlurohman Tohkimo, Inc.
Adam Rafif Faqih School of Computing, Telkom University

DOI: https://doi.org/10.33365/jatika.v6i3.496

Keywords: Requirement validation, Large language model, Software requirements

Abstract

Requirements validation is an important activity of requirements engineering. Large language models (LLMs) have been proposed to improve the efficiency of requirements validation activity. However, the LLMs validate generally without considering the stakeholders’ characteristics of the software development projects that will use the requirements. In this paper, we propose PReVAL, a novel persona-driven LLM-based validation approach that utilizes persona to reflect the characteristics of stakeholders in the prompting of LLMs. A comparative experiment demonstrates the benefits of PReVAL in terms of consistency in validating requirements based on the persona. However, introducing a persona also increases unnecessary aspects of the validation result.

Downloads

Download data is not yet available.

References

Wiegers K. and Beatty J., 2013, Software Requirements, Ed.3, Microsoft Press, Washington

The Standish Group (2020). CHAOS Report 2020. The Standish Group

Maalem, Sourour & Zarour, Nacer eddine. (2016). Challenge of validation in requirements engineering. Journal of Innovation in Digital Ecosystems. 3. 10.1016/j.jides.2016.05.001.

Maldonado, J.C., Carver, J., Shull, F. et al. Perspective-Based Reading: A Replicated Experiment Focused on Individual Reviewer Effectiveness. Empir Software Eng 11, 119–142 (2006). https://doi.org/10.1007/s10664-006-5967-6

F. Shull, I. Rus and V. Basili, "How perspective-based reading can improve requirements inspections," in Computer, vol. 33, no. 7, pp. 73-79, July 2000, doi: 10.1109/2.869376

Vogelsang, A., Fischbach, J. (2025). Using Large Language Models for Natural Language Processing Tasks in Requirements Engineering: A Systematic Guideline. In: Ferrari, A., Ginde, G. (eds) Handbook on Natural Language Processing for Requirements Engineering. Springer, Cham. https://doi.org/10.1007/978-3-031-73143-3_16

S. Lubos et al., "Leveraging LLMs for the Quality Assurance of Software Requirements," in 2024 IEEE 32nd International Requirements Engineering Conference (RE), Reykjavik, Iceland, 2024, pp. 389-397, doi: 10.1109/RE59067.2024.00046.

Cheng, Haowei & Husen, Jati & Peralta, Sien & Jiang, Bowen & Yoshioka, Nobukazu & Ubayashi, Naoyasu & Washizaki, Hironori. (2024). Generative AI for Requirements Engineering: A Systematic Literature Review. 10.48550/arXiv.2409.06741.

T. Mahbub, D. Dghaym, A. Shankarnarayanan, T. Syed, S. Shapsough and I. Zualkernan, "Can GPT-4 Aid in Detecting Ambiguities, Inconsistencies, and Incompleteness in Requirements Analysis? A Comprehensive Case Study," in IEEE Access, vol. 12, pp. 171972-171992, 2024, doi: 10.1109/ACCESS.2024.3464242.

A. Fantechi, S. Gnesi, L. Passaro and L. Semini, "Inconsistency Detection in Natural Language Requirements using ChatGPT: a Preliminary Evaluation," 2023 IEEE 31st International Requirements Engineering Conference (RE), Hannover, Germany, 2023, pp. 335-340, doi: 10.1109/RE57278.2023.00045.

Arora, C., Grundy, J., Abdelrazek, M. (2024). Advancing Requirements Engineering Through Generative AI: Assessing the Role of LLMs. In: Nguyen-Duc, A., Abrahamsson, P., Khomh, F. (eds) Generative AI for Effective Software Development. Springer, Cham. https://doi.org/10.1007/978-3-031-55642-5_6

Sun, G., Zhan, X., & Such, J. (2024). Building Better AI Agents: A Provocation on the Utilisation of Persona in LLM-based Conversational Agents. Proceedings of the 6th ACM Conference on Conversational User Interfaces.

Hu, T., & Collier, N. (2024). Quantifying the Persona Effect in LLM Simulations. ArXiv, abs/2402.10811.

Bhandari, P., Fay, N., Wise, M., Datta, A., Meek, S., Naseem, U., & Nasim, M. (2025). Can LLM Agents Maintain a Persona in Discourse?. arXiv preprint arXiv:2502.11843.

White, J., Fu, Q., Hays, S., Sandborn, M., Olea, C., Gilbert, H., Elnashar, A., Spencer-Smith, J., & Schmidt, D.C. (2023). A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT. ArXiv, abs/2302.11382.

Siddiq, M., Samee, A., Azgor, S.R., Haider, M.A., Sawraz, S.I., & Santos, J.C. (2023). Zero-shot Prompting for Code Complexity Prediction Using GitHub Copilot. 2023 IEEE/ACM 2nd International Workshop on Natural Language-Based Software Engineering (NLBSE), 56-59.

Vaswani, A., Shazeer, N.M., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., & Polosukhin, I. (2017). Attention is All you Need. Neural Information Processing Systems.

LinkedIn, " LinkedIn Developer Productivity and Happiness Framework," linkedin.com. https://github.com/linkedin/dph-framework/ (accessed Apr. 20, 2025).

Kore, A., Abbasi Bavil, E., Subasri, V., Abdalla, M., Fine, B., Dolatabadi, E., & Abdalla, M. (2024). Empirical data drift detection experiments on real-world medical imaging data. Nature Communications, 15.

Day, S. J., & Altman, D. G. (2000). Statistics notes: blinding in clinical trials and other studies. BMJ (Clinical research ed.), 321(7259), 504. https://doi.org/10.1136/bmj.321.7259.504

Stol, K., & Fitzgerald, B. (2018). The ABC of Software Engineering Research. ACM Transactions on Software Engineering and Methodology (TOSEM), 27, 1 - 51.

Ferrari A., Spagnolo O. G., and Gnesi S., 2017, PURE: A Dataset of Public Requirements Documents, Proceeding of 2017 IEEE 25thInternational Requirement Engineering Conference, Lisbon, September 2017

Heiberger, R., & Robbins, N. (2014). Design of Diverging Stacked Bar Charts for Likert Scales and Other Applications. Journal of Statistical Software, 57(5), 1–32. https://doi.org/10.18637/jss.v057.i05