artificial intelligence testing
Evaluation of Out-of-Distribution Detection Performance on Autonomous Driving Datasets
Henriksson, Jens, Berger, Christian, Ursing, Stig, Borg, Markus
Safety measures need to be systemically investigated to what extent they evaluate the intended performance of Deep Neural Networks (DNNs) for critical applications. Due to a lack of verification methods for high-dimensional DNNs, a trade-off is needed between accepted performance and handling of out-of-distribution (OOD) samples. This work evaluates rejecting outputs from semantic segmentation DNNs by applying a Mahalanobis distance (MD) based on the most probable class-conditional Gaussian distribution for the predicted class as an OOD score. The evaluation follows three DNNs trained on the Cityscapes dataset and tested on four automotive datasets and finds that classification risk can drastically be reduced at the cost of pixel coverage, even when applied on unseen datasets. The applicability of our findings will support legitimizing safety measures and motivate their usage when arguing for safe usage of DNNs in automotive perception.
- North America > United States (0.14)
- Asia > Middle East > Israel (0.05)
- Europe > Germany > Baden-Württemberg > Karlsruhe Region > Karlsruhe (0.04)
- (3 more...)
- Transportation > Ground > Road (1.00)
- Automobiles & Trucks (1.00)
Potential-based Credit Assignment for Cooperative RL-based Testing of Autonomous Vehicles
Ayvaz, Utku, Cheng, Chih-Hong, Shen, Hao
While autonomous vehicles (AVs) may perform remarkably well in generic real-life cases, their irrational action in some unforeseen cases leads to critical safety concerns. This paper introduces the concept of collaborative reinforcement learning (RL) to generate challenging test cases for AV planning and decision-making module. One of the critical challenges for collaborative RL is the credit assignment problem, where a proper assignment of rewards to multiple agents interacting in the traffic scenario, considering all parameters and timing, turns out to be non-trivial. In order to address this challenge, we propose a novel potential-based reward-shaping approach inspired by counterfactual analysis for solving the credit-assignment problem. The evaluation in a simulated environment demonstrates the superiority of our proposed approach against other methods using local and global rewards.
- Transportation > Ground > Road (0.47)
- Information Technology (0.47)
- Automobiles & Trucks (0.47)
- Leisure & Entertainment > Games (0.46)
- Information Technology > Artificial Intelligence > Robots > Autonomous Vehicles (1.00)
- Information Technology > Artificial Intelligence > Machine Learning > Reinforcement Learning (1.00)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks (0.93)
- Information Technology > Artificial Intelligence > Representation & Reasoning > Agents > Agent Societies (0.34)
Artificial Intelligence Testing (AIT)
Exams can be taken at any of the 150 Prometric test centers worldwide or online at home. Take the ISTQB accredited CT-AI online course with one of our official training providers. Looking for a more practical guide for testing AI based systems? To play, press and hold the enter key. To stop, release the enter key.
How Artificial Intelligence Testing is Top-Notch in Cyber World
In the cybersecurity sector, artificial intelligence testing is crucial. This is because AI has the potential to help cybersecurity overcome some of its major obstacles. And there are many obstacles, including the incapacity of many organizations to stay on top of the numerous new risks and attacks that emerge as the internet and technological usage increase. AI-powered cybersecurity is expected to change how we respond to cyber attacks. Because of its capacity to study and learn from enormous volumes of data, artificial intelligence will be crucial in identifying sophisticated threats.
- Information Technology > Security & Privacy (1.00)
- Government > Military > Cyberwarfare (1.00)
Technology Labs podcast: Episode 4 - Artificial Intelligence Testing
Technology leads podcast is a podcast with three hosts Tom, Daniel, and Rik. Each episode has a guest that will be interviewed. The podcast starts with a couple of interesting tech updates. We question whether these are still podcasts or more leaning towards the audiobook genre. Remote controlling your car with an app is nice, but what if you rent a car and a previous customer can still control it!
- Media > Television (0.40)
- Education > Assessment & Standards > Measuring Intelligence (0.40)
- Information Technology > Communications > Mobile (1.00)
- Information Technology > Artificial Intelligence (1.00)
An interdisciplinary approach to artificial intelligence testing - JAXenter
JAXenter: The term'intelligence' is not easy to understand. What's the best way to explain it and how can we apply it to machines? Marisa Tschopp: Human intelligence has been a very controversial topic and has undergone dramatic changes in history since the beginnings in the early 19th century. Intelligence gained importance especially in the educational context as these "mental abilities" were the best predictors for success in school and aimed to place students into the right classes. There are various, very elaborated theories, that define human intelligence.
Artificial Intelligence Testing
Neufeld, Eric Michael (University of Saskatchewan) | Finnestad, Sonje (University of Saskatchewan)
Hector Levesque has a strong critical position regarding the place of the Turing Test in Artificial Intelligence. A key argument concerns the test’s use of, or even, reliance on deception for subjectively demonstrating intelligence, and counters with a test of his own based on Winograd Schemas that he suggests is more objective. We argue that the subjectivity of the test is a strength, and that evaluating the outcome of Levesque’s objective test introduces other problems.