Grammar-AlignedDecoding

Neural Information Processing Systems 

Specifically, ingrammar-constrained decoding(GCD), the LLM'soutput must follow agiven grammar. Our algorithm uses prior sample outputs to soundly overapproximate the future grammaticality of different output prefixes.