Cheaply Estimating Inference Efficiency Metrics for Autoregressive Transformer Models

Jan-19-2025, 23:12:30 GMT–Neural Information Processing Systems 

Large language models (LLMs) are highly capable but also computationally expensive.