AutoPentester: An LLM Agent-based Framework for Automated Pentesting

This paper is submitted to the IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom) 2025 and may be revised following the review process. Abstract--Penetration testing and vulnerability assessment are essential industry practices for safeguarding computer systems. As cyber threats grow in scale and complexity, the demand for pentesting has surged, surpassing the capacity of human professionals to meet it effectively. With advances in AI, particularly Large Language Models (LLMs), there have been attempts to automate the pentesting process. However, existing tools such as PentestGPT are still semi-manual, requiring significant professional human interaction to conduct pentests. T o this end, we propose a novel LLM agent-based framework, AutoPen-tester, which automates the pentesting process. Given a target IP, AutoPentester automatically conducts pentesting steps using common security tools in an iterative process. It can dynamically generate attack strategies based on the tool outputs from the previous iteration, mimicking the human pentester approach. We evaluate AutoPentester using Hack The Box and custom-made VMs, comparing the results with the state-of-the-art PentestGPT . Results show that AutoPentester achieves a 27.0% better subtask completion rate and 39.5% more vulnerability coverage with fewer steps. Most importantly, it requires significantly fewer human interactions and interventions compared to PentestGPT . Furthermore, we recruit a group of security industry professional volunteers for a user survey and perform a qualitative analysis to evaluate AutoPentester against industry practices and compare it with PentestGPT . On average, AutoPentester received a score of 3.93 out of 5 based on user reviews, which was 19.8% higher than PentestGPT . Cyber incidents and attacks, whether they are data breaches, ransomware, espionage, phishing, or business email compromises, are increasing at an alarming rate globally [1]. Instead, they now focus on SMEs [2], public sectors, and even essential services such as hospitals [3] and emergency services, in addition to individuals. This has created an environment where every corporate computer system, irrespective of its scale, needs to be secured and maintain a good security posture.