On-Premise SLMs vs. Commercial LLMs: Prompt Engineering and Incident Classification in SOCs and CSIRTs

In this study, we evaluate open-source models for security incident classification, comparing them with proprietary models. We utilize a dataset of anonymized real incidents, categorized according to the NIST SP 800-61r3 taxonomy and processed using five prompt-engineering techniques (PHP, SHP, HTP, PRP, and ZSL). The results indicate that, although proprietary models still exhibit higher accuracy, locally deployed open-source models provide advantages in privacy, cost-effectiveness, and data sovereignty. According to CERT.br, Brazil reported over 516k security incidents in 2024 and more than 181k in the first half of 2025, underscoring a persistent upward trend that challenges SOCs and CSIRTs to manage high alert volumes efficiently [1]. To alleviate this overload, AI-driven solutions, particularly prompt-engineering techniques such as Progressive Hint Prompting (PHP), have demonstrated over 90% accuracy with models like GPT -4o and Gemini 2 [2].