Goto

Collaborating Authors

 Government


Impacts of floating-point non-associativity on reproducibility for HPC and deep learning applications

arXiv.org Artificial Intelligence

Run-by-run variability in parallel programs caused by floating-point non-associativity (FPNA) has been known to significantly affect reproducibility in iterative algorithms, due to accumulating errors. Non-reproducibility negatively affects efficiency and effectiveness of correctness testing for stochastic programs. Recently, the sensitivity of deep learning (DL) training and inference pipelines to FPNA have been found to be extreme, and can prevent certification for commercial applications, accurate assessment of robustness and sensitivity, and bug detection. New approaches in scientific computing applications have coupled DL models with high-performance computing (HPC) simulations, leading to an aggravation of debugging and testing challenges. Here we perform an investigation of the statistical properties of FPNA within modern parallel programming models, analyze performance and productivity impacts of replacing atomic operations with deterministic alternatives on GPUs, and examine the recently-added deterministic options within the PyTorch framework within the context of GPU deployment, uncovering and quantifying the impacts of input parameters triggering run-by-run variability and reporting on the reliability and completeness of the documentation. Finally, we evaluate the strategy of exploiting automatic determinism provided by deterministic hardware, using the Groq LPU$^{TM}$ accelerator for inference portions of the DL pipeline. We demonstrate the benefits that this strategy can provide within reproducibility and correctness efforts.


Dynamic Label Adversarial Training for Deep Learning Robustness Against Adversarial Attacks

arXiv.org Artificial Intelligence

Adversarial training is one of the most effective methods for enhancing model robustness. Recent approaches incorporate adversarial distillation in adversarial training architectures. However, we notice two scenarios of defense methods that limit their performance: (1) Previous methods primarily use static ground truth for adversarial training, but this often causes robust overfitting; (2) The loss functions are either Mean Squared Error or KL-divergence leading to a sub-optimal performance on clean accuracy. To solve those problems, we propose a dynamic label adversarial training (DYNAT) algorithm that enables the target model to gradually and dynamically gain robustness from the guide model's decisions. Additionally, we found that a budgeted dimension of inner optimization for the target model may contribute to the trade-off between clean accuracy and robust accuracy. Therefore, we propose a novel inner optimization method to be incorporated into the adversarial training. This will enable the target model to adaptively search for adversarial examples based on dynamic labels from the guiding model, contributing to the robustness of the target model.


Which Prosodic Features Matter Most for Pragmatics?

arXiv.org Artificial Intelligence

We investigate which prosodic features matter most in conveying prosodic functions. We use the problem of predicting human perceptions of pragmatic similarity among utterance pairs to evaluate the utility of prosodic features of different types. We find, for example, that duration-related features are more important than pitch-related features, and that utterance-initial features are more important than utterance-final features. Further, failure analysis indicates that modeling using pitch features only often fails to handle important pragmatic functions, and suggests that several generally-neglected acoustic and prosodic features are pragmatically significant, including nasality and vibrato. These findings can guide future basic research in prosody, and suggest how to improve speech synthesis evaluation, among other applications.


Personalised Medicine: Establishing predictive machine learning models for drug responses in patient derived cell culture

arXiv.org Artificial Intelligence

The concept of personalised medicine in cancer therapy is becoming increasingly important. There already exist drugs administered specifically for patients with tumours presenting well-defined mutations. However, the field is still in its infancy, and personalised treatments are far from being standard of care. Personalised medicine is often associated with the utilisation of omics data. Yet, implementation of multi-omics data has proven difficult, due to the variety and scale of the information within the data, as well as the complexity behind the myriad of interactions taking place within the cell. An alternative approach to precision medicine is to employ a function-based profile of the cell. This involves screening a range of drugs against patient derived cells. Here we demonstrate a proof-of-concept, where a collection of drug screens against a highly diverse set of patient-derived cell lines, are leveraged to identify putative treatment options for a 'new patient'. We show that this methodology is highly efficient in ranking the drugs according to their activity towards the target cells. We argue that this approach offers great potential, as activities can be efficiently imputed from various subsets of the drug treated cell lines that do not necessarily originate from the same tissue type.


Optimally Solving Simultaneous-Move Dec-POMDPs: The Sequential Central Planning Approach

arXiv.org Artificial Intelligence

Centralized training for decentralized execution paradigm emerged as the state-of-the-art approach to epsilon-optimally solving decentralized partially observable Markov decision processes. However, scalability remains a significant issue. This paper presents a novel and more scalable alternative, namely sequential-move centralized training for decentralized execution. This paradigm further pushes the applicability of Bellman's principle of optimality, raising three new properties. First, it allows a central planner to reason upon sufficient sequential-move statistics instead of prior simultaneous-move ones. Next, it proves that epsilon-optimal value functions are piecewise linear and convex in sufficient sequential-move statistics. Finally, it drops the complexity of the backup operators from double exponential to polynomial at the expense of longer planning horizons. Besides, it makes it easy to use single-agent methods, e.g., SARSA algorithm enhanced with these findings applies while still preserving convergence guarantees. Experiments on two- as well as many-agent domains from the literature against epsilon-optimal simultaneous-move solvers confirm the superiority of the novel approach. This paradigm opens the door for efficient planning and reinforcement learning methods for multi-agent systems.


Localized Observation Abstraction Using Piecewise Linear Spatial Decay for Reinforcement Learning in Combat Simulations

arXiv.org Artificial Intelligence

In the domain of combat simulations, the training and deployment of deep reinforcement learning (RL) agents still face substantial challenges due to the dynamic and intricate nature of such environments. Unfortunately, as the complexity of the scenarios and available information increases, the training time required to achieve a certain threshold of performance does not just increase, but often does so exponentially. This relationship underscores the profound impact of complexity in training RL agents. This paper introduces a novel approach that addresses this limitation in training artificial intelligence (AI) agents using RL. Traditional RL methods have been shown to struggle in these high-dimensional, dynamic environments due to real-world computational constraints and the known sample inefficiency challenges of RL. To overcome these limitations, we propose a method of localized observation abstraction using piecewise linear spatial decay. This technique simplifies the state space, reducing computational demands while still preserving essential information, thereby enhancing AI training efficiency in dynamic environments where spatial relationships are often critical. Our analysis reveals that this localized observation approach consistently outperforms the more traditional global observation approach across increasing scenario complexity levels. This paper advances the research on observation abstractions for RL, illustrating how localized observation with piecewise linear spatial decay can provide an effective solution to large state representation challenges in dynamic environments.


How far can Ukraine's military go inside Russia?

Al Jazeera

Moscow has come under one of the largest drone attacks of the war.Read more When President Vladimir Putin launched Russia's so-called "special military operation" in Ukraine two-and-a-half years ago, he expected a speedy victory. Not only did that not happen, but Ukraine has now brought the war home to Russia. Russia faces manpower woes after failing to stop Ukraine's Kursk incursion list 2 of 4 Russians flock to evacuation centre to flee Ukraine's incursion in Kursk list 4 of 4 The capital has faced one of its biggest drone attacks of the war – according to the mayor of Moscow. Meanwhile, Ukraine's incursion into the Kursk region caught Russia by surprise. Has Ukraine's bold move put on hold discussions about a stalemate and possible negotiations involving concessions to Russia? What are the prospects for a Gaza ceasefire deal?


DeepMind workers urge Google to drop military contracts

Engadget

Google DeepMind workers have signed a letter calling on the company to drop contracts with military organizations, according to a report by Time. The document was drafted on May 16 of this year. Around 200 people signed the document, which amounts to five percent of the total headcount of DeepMind. For the uninitiated, DeepMind is one of Google's AI divisions and the letter states that adopting military contracts runs afoul of the company's own AI rules. The letter was sent out as internal concerns began circulating within the AI lab that the tech was allegedly being sold to military organizations via cloud contracts.


What is Israel doing to Palestinians in Tulkarem?

Al Jazeera

Israel killed three Palestinians in a drone strike on Thursday in Tulkarem, a city and refugee camp in the occupied West Bank. That was during an Israeli raid – a near-daily occurrence in the West Bank – on the Tulkarem refugee camp, during which Israeli troops clashed with fighters from the Qassam Brigades, the military wing of Hamas, according to fighters in the city. Here's all you need to know about Israeli raids on Tulkarem: News reports say Israeli soldiers were deployed on rooftops and sent bulldozers into the camp to destroy large residential areas. Israel also reportedly set fire to people's homes and prevented local relief workers from putting the fires out. Experts say Israel's tactics during its raids appear to be part of a broader doctrine to collectively punish the population, ostensibly because pockets of armed resistance are fighting back against Israel's ever-entrenching occupation. Israel claims that it is conducting "counter-terrorism" operations.


Wyoming man running as bot concedes race, launches 'alliance' to inject AI into politics

FOX News

Wyoming man Victor Miller, who filed mayoral candidacy as AI bot "VIC," speaks out after OpenAI shuts down his account. Victor Miller, who had been running as an artificial intelligence-powered bot named "VIC" [Virtual Integrated Citizen] in Wyoming's capital city, conceded his bid to make technological political history on Wednesday. Miller received 327 votes, or about 3% of the total cast, in Cheyenne's nonpartisan mayoral primary on Tuesday night, according to Laramie County records. On Wednesday, Fox News Digital obtained a statement from Miller saying that he and VIC came up short in their bid to change the definition of political machine in the Cowboy State's capital city: "Today, I, Victor Miller, concede the Cheyenne mayoral race. As the first person to put artificial intelligence directly on the ballot, offering voters the novel choice of AI governance, our campaign has marked a historic moment in politics and technology," Miller said.