Existing LMs struggle with proof-oriented programming due to data scarcity, which manifest in two key ways: (1) a lack of sufficient corpora for proof-oriented programming languages such as F*, and (2) the absence of large-scale, project-level proof-oriented implementations that can teach the model the intricate reasoning process when performing proof-oriented programming. We present the first on synthetic data augmentation for project level proof oriented programming for both generation and repair. Our method addresses data scarcity by synthesizing basic proof-oriented programming problems for proficiency in that language; incorporating diverse coding data for reasoning capability elicitation and creating new proofs and repair data within existing repositories. This approach enables language models to both synthesize and repair proofs for function- and repository-level code. We show that our fine-tuned 14B parameter model, PoPilot, can exceed the performance of the models that outperforms GPT-4o in project-level proof-oriented programming by 64% relative margin, and can improve GPT-4o's performance by 54% by repairing its outputs over GPT-4o's self-repair.

As an avid reader, you will realize I am not a big proponent of Artificial Intelligence (AI). First, not just because, these days, every line of code is referred to as AI. Or a machine with a few sensors is suddenly called a robot, magically suggesting superior qualities of adaptability. The second major gripe I have with AI is how artificial the definition of intelligence is in AI. For intelligence is knowing what to do in unprecedented scenarios no AI system provides today.

When human experts build qualitative or quantitative models of complex systems, they use the function of the system as a guideline to decide what to model and how to model it, yet they do not often encode this functional knowledge directly. If qualitative and quantitative models contained this functional knowledge, our reasoning systems might use it as a heuristic or as a filter during the course of quantitative and qualitative simulation. Matthew Klenk (PARC) delivered a separate talk related to massive-scale model-based reasoning, describing the challenge of choosing initial conditions for simulation. Throughout the technical presentations on advances in qualitative simulation, we discussed the practicality of automatically transforming quantitative and qualitative models during the course of reasoning.