Auto-Formulating Dynamic Programming Problems with Large Language Models

Automating the formulation of decision-making problems represents a major step toward fully autonomous decision-support systems. Traditionally, solving such problems involves two sequential stages: first, translating real-world scenarios into well-defined mathematical models-an essential skill emphasized in operations research education-and second, applying computational tools to find optimal or near-optimal solutions. While substantial research in recent decades has primarily focused on the second stage-enhancing algorithms and improving solver efficiency-advancements span a wide range, from foundational developments such as reinforcement learning (RL) frameworks (e.g., Sutton and Barto 2018) and approximate dynamic programming techniques (e.g., Powell 2011), to powerful solvers like COPT, CPLEX, and Gurobi. Such innovations coupled with increasing computational power have led to high-impact real-world applications, exemplified by AlphaGo, which leveraged deep learning and RL to solve complex, large-scale decision-making problems (Silver et al. 2016). That said, while these advancements have shifted many computational tasks to automated software, the initial problem formulation step has largely remained manual and dependent on expert knowledge. The recent rapid progress in large language models (LLMs) provides a promising opportunity to automate this crucial first step. LLMs excel in natural language processing and have demonstrated significant potential for effectively automating the formulation of mathematical models directly from plain English descriptions. Leveraging LLMs can substantially reduce the human expertise required, simplify the problem formulation process, and make advanced optimization methods accessible to a broader audience. Among various optimization problems, dynamic programming (DP) represents a particularly important yet challenging category for formulation automation.