Goto

Collaborating Authors

 test driver


The Foundation Cracks: A Comprehensive Study on Bugs and Testing Practices in LLM Libraries

arXiv.org Artificial Intelligence

Large Language Model (LLM) libraries have emerged as the foundational infrastructure powering today's AI revolution, serving as the backbone for LLM deployment, inference optimization, fine-tuning, and production serving across diverse applications. Despite their critical role in the LLM ecosystem, these libraries face frequent quality issues and bugs that threaten the reliability of AI systems built upon them. To address this knowledge gap, we present the first comprehensive empirical investigation into bug characteristics and testing practices in modern LLM libraries. We examine 313 bug-fixing commits extracted across two widely-adopted LLM libraries: HuggingFace Transformers and vLLM.Through rigorous manual analysis, we establish comprehensive taxonomies categorizing bug symptoms into 5 types and root causes into 14 distinct categories.Our primary discovery shows that API misuse has emerged as the predominant root cause (32.17%-48.19%), representing a notable transition from algorithm-focused defects in conventional deep learning frameworks toward interface-oriented problems. Additionally, we examine 7,748 test functions to identify 7 distinct test oracle categories employed in current testing approaches, with predefined expected outputs (such as specific tensors and text strings) being the most common strategy. Our assessment of existing testing effectiveness demonstrates that the majority of bugs escape detection due to inadequate test cases (41.73%), lack of test drivers (32.37%), and weak test oracles (25.90%). Drawing from these findings, we offer some recommendations for enhancing LLM library quality assurance.


Apple's Mystery Self-Driving Car Tech Covered 20,921 KM Last Year

#artificialintelligence

Rumours of the possible existence of an Apple car have been making rounds for ages. Over the years, various executives have come and gone, and several carmakers have been tied to the project. And now, there's further proof that Apple is developing systems for a car thanks to new filings from the California Department of Motor Vehicles (DMV). In order to develop its vision for a car of the future, Apple has been testing its automotive driving tech on the streets of California. As part of the testing of these systems, the firm is required to log all miles covered by self-driving cars with the State of California. The firm does this alongside companies like Waymo and Cruise, which are also developing self-driving cars.


Ford recruits two robot drivers for testing in its 'weather factory'

Daily Mail - Science & tech

Ford is using two robotic test drivers โ€“ affectionately named Shelby and Miles โ€“ to trial its vehicles in extreme temperatures. The robots are conducting tests in environmental conditions that are too treacherous for any human worker to endure. Shelby and Miles can operate at temperatures ranging from -40 F to 176 F (-40 C to 80 C) as well as at extreme altitudes, Ford says. Their robotic legs extend to the accelerator, brake and clutch pedals, with one arm positioned to change gear and the other used to start and stop the engine. The tests are taking place at Ford's secretive'weather factory' in Cologne, Germany โ€“ a building the size of a football pitch that's dedicated to R&D work.


First Guidelines For Robocar Test Drivers

#artificialintelligence

Experimental self-driving car, based on modified Ford automobile, with Lidar and other sensors ... [ ] visible, in the Mission Bay neighborhood of San Francisco, California, June 10, 2019. Testing self-driving vehicles on public roads remains a scary prospect for citizens of communities where that's happening but a six-month old consortium of major automakers and ride-share companies has taken a step towards removing some of that fear, by addressing the human element. An operator sits in the driver's seat of a Toyota Motor Corp. Prius hybrid car, operated by ... [ ] Yandex.Taxi, part of Yandex.NV, during a self-driving taxi trial on open roads in Moscow, Russia, on Wednesday, Aug. 21, 2019. Yandex, Russia's largest search engine that successfully expanded to online taxi and swallowed Uber Technologies Inc. operations in the country, started testing self-driving cars in 2017. The humans are what's known as in-vehicle fallback test drivers.


Deep learning & the human brain - Imaginea

#artificialintelligence

Deep learning also uses deduction, but in a linear, basic, and one-dimensional way. Training the artificial neural networks to classify lions as dangerous might make them sensitive only to lions. A bear can't get classified as dangerous automatically. Training them to identify a cat will only make them recognize a cat, but not deduce that a leopard belongs to the cat family. Similarly, through facial recognition, deep learning can tag faces on photos but might stumble when there are faces of siamese twins.


Uber death leaves questions about self-driving car liability unanswered

#artificialintelligence

A year after the first fatality caused by a fully self-driving car, questions about liability in the event of a death involving the cars are still completely up in the air. Officials announced earlier this week that Uber won't face criminal charges in the death of a pedestrian struck and killed by one of its self-driving cars nearly a year ago in Tempe, Arizona. The Yavapai County Attorney's Office said it conducted a thorough review of the evidence and determined there was no basis for criminal liability against Uber. It did not detail how the decision was made and has declined to answer any questions on the case. The pedestrian was walking a bicycle across a road at night.


Uber to Resume Self-Driving Car Tests in Pittsburgh After Hiatus

WSJ.com: WSJD - Technology

Uber had previously been pushing to get a fully autonomous vehicle on the roads by year-end, part of its mad dash to keep abreast of rival Alphabet Inc. GOOGL -3.16% But those plans were derailed when an Uber car struck and killed a pedestrian in Tempe, Ariz., in a March incident that some safety experts have said was avoidable. Uber on Thursday will send out just a handful of the self-driving vehicles on a fixed one-mile route between two of its offices in Pittsburgh. It is home to the company's largest testing and development center and one of the three cities where it had been experimenting with the cars. The runs are limited to the daytime during weekdays and speeds no more than 25 miles an hour.


Ex-Google driverless car firm Waymo begins charging for self-driving car rides in Arizona

USATODAY - Tech Top Stories

Waymo won't offer a public ride service, but will charge pre-approved riders for its autonomous vans through a program called Waymo One. Traffic is shown behind a Waymo self-driving car Nov. 28, 2018, during a demonstration ride in Chandler. The former Google driverless car company will begin charging pre-approved passengers in the Chandler, Arizona, area for rides on its self-driving ride-sharing service Wednesday. Waymo officials said the robot cars will be offered to the general public "over time," but initially only a limited number of people screened and invited by the company will be able to hail a ride from the new Waymo One commercial service. The company had previously said that it would launch a public service by the end of 2018 in the metro Phoenix area.


Waymo Collision Illustrates Why Society Might Eventually Ban Human Driving

#artificialintelligence

On October 19, a Waymo Pacifica struck and injured a motorcyclist in California. As is often the case, the collision was caused by a human - in this instance, the safety driver in the Waymo vehicle. In an unusual twist, however, Waymo CEO John Krafcik revealed that if the safety operator had not taken control of the autonomous minivan, then the self-driving software would have avoided a collision. Our simulation shows the self-driving system would have responded to the passenger car by reducing our vehicle's speed, and nudging slightly in our own lane, avoiding a collision." Waymo Autonomous Vehicle ("WaymoAV") was traveling at approximately 21 MPH westbound in Lane 2 of El Camino Real in Mountain View in self-driving mode. A passenger vehicle in Lane 1, to the left of the Waymo AV, began to change lanes into Lane2 to avoid a box truck blocking two lanes of traffic, Waymo's test driver took manual control of the AV out of an abundance of caution, disengaged from self-driving mode, and began changing lanes into Lane 3. A motorcycle, traveling at approximately 28 MPH behind the Waymo AV, had just entered Lane 3 to overtake the Waymo AV on its right. The motorcyclist reported injuries and was transported to the hospital for treatment. The Waymo AV sustained minor damage to the rear bumper."


Uber Cuts More Test Drivers in Its Robot Car Operation

WSJ.com: WSJD - Technology

Uber has been doing an extensive review of the self-driving vehicle program since the Tempe pedestrian was struck and killed as she crossed the street. Investigators later said the sole vehicle operator, whose job was to be a fail-safe for when the cars act erratically, was streaming a television show on her phone, contributing to the accident. Self-driving vehicle experts have questioned the effectiveness of Uber's technology and its move in late 2017 to stop using a second safety driver that it had previously stationed in the front passenger seat of its autonomous test cars. The company had disabled an automatic emergency braking system on the Volvo involved in the crash, which put added responsibility on the operator to stop or swerve the vehicle in the case of an obstacle, according to a report from federal investigators last month. Many technologists view self-driving vehicles as the future of transportation because they eliminate human error, thereby reducing costs from crashes and liability insurance.