Current agentic AI benchmarks predominantly evaluate task completion accuracy, while overlooking critical enterprise requirements such as cost-efficiency, reliability, and operational stability. Through systematic analysis of 12 main benchmarks and empirical evaluation of state-of-the-art agents, we identify three fundamental limitations: (1) absence of cost-controlled evaluation leading to 50x cost variations for similar precision, (2) inadequate reliability assessment where agent performance drops from 60\% (single run) to 25\% (8-run consistency), and (3) missing multidimensional metrics for security, latency, and policy compliance. We propose \textbf{CLEAR} (Cost, Latency, Efficacy, Assurance, Reliability), a holistic evaluation framework specifically designed for enterprise deployment. Evaluation of six leading agents on 300 enterprise tasks demonstrates that optimizing for accuracy alone yields agents 4.4-10.8x more expensive than cost-aware alternatives with comparable performance. Expert evaluation (N=15) confirms that CLEAR better predicts production success (correlation $ρ=0.83$) compared to accuracy-only evaluation ($ρ=0.41$).

Neural Information Processing Systems http://nips.cc/

Our approach reduces the problem into a sequence of plug-in classifier learning tasks.

Our approach reduces the problem into a sequence of plug-in classifier learning tasks.

We greatly appreciate the reviewers for taking a close look at the paper and the proofs, and giving a detailed feedback. We first address the common concerns raised. This is the same dependence that the previous method of Narasimhan (2018) incurs [26]. We'll definitely include this in the appendix. We'll include these details in the paper, along with an example.

We present a general framework for solving a large class of learning problems with non-linear functions of classification rates.

Our approach reduces the problem into a sequence of plug-in classifier learning tasks.