Instruction-Following Evaluation in Function Calling for Large Language Models

Function calling is a core capability of Large Language Models (LLMs), essential for AI agents. We introduce IFEval-FC, a benchmark inspired by IFEval (Zhou et al., 2023), which assesses precise instruction following in function calling. IFEval-FC encodes verifiable formats directly within JSON schema descriptions, such as "a value must not contain punctuation". It offers 750 test cases, each consisting of a function with an embedded format for one of its input parameters and a corresponding user query. The evaluation is fully algorithmic, ensuring objectivity, reproducibility, and scalability. Our results indicate that even state-of-the-art proprietary models, such as GPT -5 (OpenAI, 2025) and Claude Opus 4.1 (Anthropic, 2025), frequently fail to adhere to basic formatting rules, highlighting a significant limitation for practical applications in real-world agent systems.