On Automating Security Policies with Contemporary LLMs