Supporting Human-AI Collaboration in Auditing LLMs with LLMs