A Tale of LLMs and Induced Small Proxies: Scalable Agents for Knowledge Mining