Language model (LM) agents have gained significant attention for their ability to autonomously complete tasks through interactions with environments, tools, and APIs. LM agents are primarily built with prompt engineering or supervised finetuning. At the same time, reinforcement learning (RL) has been explored to enhance LM's capabilities, such as reasoning and factuality. However, the combination of the LM agents and reinforcement learning (Agent-RL) remains underexplored and lacks systematic study. To this end, we built AgentFly, a scalable and extensible Agent-RL framework designed to empower LM agents with a variety of RL algorithms. Our framework supports multi-turn interactions by adapting traditional RL methods with token-level masking. It features a decorator-based interface for defining tools and reward functions, enabling seamless extension and ease of use. To support high-throughput training, we implement asynchronous execution of tool calls and reward computations, and design a centralized resource management system for scalable environment coordination. We also provide a suite of prebuilt tools and environments, demonstrating the framework's effectiveness through successful agent training across multiple tasks.

The Next-Generation Transportation program coordinates the evolution and transformation of the current air-traffic management (ATM) system for the National Airspace System (NAS). Currently the NAS has a limited capacity and cannot handle the increasing future air traffic demands. However, before newly proposed ATM concepts are deployed they must be rigorously evaluated under realistic conditions. This paper presents AGENTFLY, an emerging NAS-wide highfidelity multi-agent ATM simulator with precise emulation of the human controller operation workload model and human-system interaction. The simulator is validated using a flight scenario developed by the U.S. Federal Aviation Administration that is based on real data. We present preliminary results focusing on the accuracy of the simulated controllers within AGENTFLY.