LinguaSim: Interactive Multi-Vehicle Testing Scenario Generation via Natural Language Instruction Based on Large Language Models

This layer contains the information of the background adversarial vehicles whose behaviors are not directly guided by LinguaSim. These vehicles are automatically generated and placed around the ego vehicle and the guided adversarial vehicles by LLM agent Chaos Maker, and roam aimlessly on the given map. The background vehicles significantly increase the uncertainty and complexity of the generated scenarios. B. Adversarial Behavior Generation Compared to other state-of-the-art methods for generating 3D realistic scenarios from natural language descriptions, LinguaSim achieves a higher level of realism, flexibility, and interactivity due to the innovative structure of its Action Generator agent. The detailed workflow of this component will be elaborated further in this section, with a simplified operational logic of the Action Generator illustrated in Figure 1. Figure 1: The basic workflow of module Action Generator To establish a solid foundation for the Action Generator, a retrieval database was constructed to store various behaviors available for the guided adversarial vehicles. Each behavior in the database is referred to as an Atomic Behavior, serving as a fundamental component in the subsequent process. As illustrated in Figure 1, each Atomic Behavior comprises three essential parts: 1) Agent Selection: An autonomous driving agent is selected to guide the adversarial vehicle to which Figure 1: An example of the Behavior T opology W eb generated by the Action Generator the Atomic Behavior is applied. LinguaSim includes various predefined agents, such as the basic CARLA built-in agent that follows a given route, an auto cruise control (ACC) agent that follows the vehicle in front, or the PlanT agent, an imitation-learning-based planning algorithm developed by Renz, Chitta et al. [10]. These agents serve different purposes; for example, the F ollow V ehicle behavior uses the ACC agent, while the PlanT agent is often used for less aggressive behaviors to mimic cautious drivers.