Why is constrained neural language generation particularly challenging?

Recent advances in deep neural language models combined wit h the capacity of large scale datasets have accelerated the development of natural langu age generation systems that produce fluent and coherent texts (to various degrees of succ ess) in a multitude of tasks and application contexts. However, controlling the output of t hese models for specific user and task needs is still an open challenge. This is crucial not onl y to customizing the content and style of the generated language, but also to their safe and re liable deployment in the real world. We present an extensive survey on the emerging topic o f constrained neural language generation in which we formally define and categorize the pro blems of natural language generation by distinguishing between conditions and constraints (the latter being testable conditions on the output text instead of the input), present constrained text generation tasks, and review existing methods and evaluation metrics for cons trained text generation. Our aim is to highlight recent progress and trends in this emergi ng field, informing on the most promising directions and limitations towards advancing th e state-of-the-art of constrained neural language generation research.