Neural Information Processing Systems http://nips.cc/

"It brings accountability to the industry," says Raquel Urtasun, Waabi's firebrand founder and CEO (who is also a professor at the University of Toronto). "There are no more excuses." After quitting Uber, where she led the ride-sharing firm's driverless-car division, Urtasun founded Waabi in 2021 with a different vision for how autonomous vehicles should be made. The firm, which has partnerships with Uber Freight and Volvo, has been running real trucks on real roads in Texas since 2023, but it carries out the majority of its development inside a simulation called Waabi World. Waabi is now taking its sim-first approach to the next level, using Waabi World not only to train and test its driving models but to prove their real-world safety.

Large-scale data is crucial for learning realistic and capable driving policies. However, it can be impractical to rely on scaling datasets with real data alone. The majority of driving data is uninteresting, and deliberately collecting new long-tail scenarios is expensive and unsafe. We propose asymmetric self-play to scale beyond real data with additional challenging, solvable, and realistic synthetic scenarios. Our approach pairs a teacher that learns to generate scenarios it can solve but the student cannot, with a student that learns to solve them. When applied to traffic simulation, we learn realistic policies with significantly fewer collisions in both nominal and long-tail scenarios. Our policies further zero-shot transfer to generate training data for end-to-end autonomy, significantly outperforming state-of-the-art adversarial approaches, or using real data alone. For more information, visit https://waabi.ai/selfplay .

The tasks of object detection and trajectory forecasting play a crucial role in understanding the scene for autonomous driving. These tasks are typically executed in a cascading manner, making them prone to compounding errors. Furthermore, there is usually a very thin interface between the two tasks, creating a lossy information bottleneck. To address these challenges, our approach formulates the union of the two tasks as a trajectory refinement problem, where the first pose is the detection (current time), and the subsequent poses are the waypoints of the multiple forecasts (future time). To tackle this unified task, we design a refinement transformer that infers the presence, pose, and multi-modal future behaviors of objects directly from LiDAR point clouds and high-definition maps. We call this model DeTra, short for object Detection and Trajectory forecasting. In our experiments, we observe that \ourmodel{} outperforms the state-of-the-art on Argoverse 2 Sensor and Waymo Open Dataset by a large margin, across a broad range of metrics. Last but not least, we perform extensive ablation studies that show the value of refinement for this task, that every proposed component contributes positively to its performance, and that key design choices were made.

While autonomous driving has long relied on machine learning to plan routes and detect objects, some companies and researchers are now betting that generative AI -- models that take in data of their surroundings and generate predictions -- will help bring autonomy to the next stage. Wayve, a Waabi competitor, released a comparable model last year that is trained on the video that its vehicles collect. Waabi's model works in a similar way to image or video generators like OpenAI's DALL-E and Sora. It takes point clouds of lidar data, which visualize a 3D map of the car's surroundings, and breaks them into chunks, similar to how image generators break photos into pixels. Based on its training data, Copilot4D then predicts how all points of lidar data will move.

The goal of this paper is to generate high-quality 3D object proposals in the context of autonomous driving. Our method exploits stereo imagery to place proposals in the form of 3D bounding boxes. We formulate the problem as minimizing an energy function encoding object size priors, ground plane as well as several depth informed features that reason about free space, point cloud densities and distance to the ground. Our experiments show significant performance gains over existing RGB and RGB-D object proposal methods on the challenging KITTI benchmark. Combined with convolutional neural net (CNN) scoring, our approach outperforms all existing results on all three KITTI object classes.

Artificial intelligence is widely expected to transform our lives. Leaders from across the sector gathered for a TIME dinner conversation on Nov. 30, where they emphasized the need to center humans in decisions around incorporating the technology into workflows and advocated for governments and industry leaders to take a responsible approach to managing the risks the technology poses. As part of the TIME100 Talks series in San Francisco, senior correspondent Alice Park spoke with panelists Cynthia Breazeal, a pioneer in social robotics and the Dean for Digital Learning at MIT, James Landay, a computer science professor and vice director of the Institute for Human-Centered AI at Stanford University, and Raquel Urtasun, CEO and founder of self-driving tech startup Waabi, which recently put a fleet of trucks into service on Uber Freight's trucking network. The panelists discussed the ethical considerations of AI and the ways in which leaders can ensure its benefits reach every corner of the world. During the discussion, the three panelists highlighted the transformative journey of AI and delved into its profound implications, emphasizing the need for responsible AI deployment.

Self-driving vehicles (SDVs) must be rigorously tested on a wide range of scenarios to ensure safe deployment. The industry typically relies on closed-loop simulation to evaluate how the SDV interacts on a corpus of synthetic and real scenarios and verify it performs properly. However, they primarily only test the system's motion planning module, and only consider behavior variations. It is key to evaluate the full autonomy system in closed-loop, and to understand how variations in sensor data based on scene appearance, such as the shape of actors, affect system performance. In this paper, we propose a framework, Adv3D, that takes real world scenarios and performs closed-loop sensor simulation to evaluate autonomy performance, and finds vehicle shapes that make the scenario more challenging, resulting in autonomy failures and uncomfortable SDV maneuvers. Unlike prior works that add contrived adversarial shapes to vehicle roof-tops or roadside to harm perception only, we optimize a low-dimensional shape representation to modify the vehicle shape itself in a realistic manner to degrade autonomy performance (e.g., perception, prediction, and motion planning). Moreover, we find that the shape variations found with Adv3D optimized in closed-loop are much more effective than those in open-loop, demonstrating the importance of finding scene appearance variations that affect autonomy in the interactive setting.

We present Sim-on-Wheels, a safe, realistic, and vehicle-in-loop framework to test autonomous vehicles' performance in the real world under safety-critical scenarios. Sim-on-wheels runs on a self-driving vehicle operating in the physical world. It creates virtual traffic participants with risky behaviors and seamlessly inserts the virtual events into images perceived from the physical world in real-time. The manipulated images are fed into autonomy, allowing the self-driving vehicle to react to such virtual events. The full pipeline runs on the actual vehicle and interacts with the physical world, but the safety-critical events it sees are virtual. Sim-on-Wheels is safe, interactive, realistic, and easy to use. The experiments demonstrate the potential of Sim-on-Wheels to facilitate the process of testing autonomous driving in challenging real-world scenes with high fidelity and low risk.

The self-driving era is here, just not the one that was promised. Instead of sleek pods without steering wheels ready to chauffeur buyers off the lot, there are mostly driverless Chevy compacts, Chrysler minivans, and Ford box trucks with bolted-on hardware trundling around bits of the U.S. southwest and, as of August, a short loop of roads in Ontario. But while the current reality has fallen far short of automaker predictions, it's worth stopping to acknowledge that there are trucks driving around public Canadian roads making deliveries, without a soul inside. The technological achievement of the feat has huge implications for business, and society, but the latest industry outlook, humbled by past failures, points to a more gradual rollout. "People think there will be a magic day where suddenly everything will be autonomous, but that's not how this is going to work," said Raquel Urtasun, a leading artificial intelligence researcher and chief executive of Toronto-based autonomous outfit Waabi Innovation Inc. "You will have certain areas where this technology is going to deploy, and then those areas will expand under more and more difficult situations."