The Synergy of Speculative Decoding and Batching in Serving Large Language Models

Open in new window