Government
Towards Scalable and Robust Model Versioning
Ding, Wenxin, Bhagoji, Arjun Nitin, Zhao, Ben Y., Zheng, Haitao
As the deployment of deep learning models continues to expand across industries, the threat of malicious incursions aimed at gaining access to these deployed models is on the rise. Should an attacker gain access to a deployed model, whether through server breaches, insider attacks, or model inversion techniques, they can then construct white-box adversarial attacks to manipulate the model's classification outcomes, thereby posing significant risks to organizations that rely on these models for critical tasks. Model owners need mechanisms to protect themselves against such losses without the necessity of acquiring fresh training data - a process that typically demands substantial investments in time and capital. In this paper, we explore the feasibility of generating multiple versions of a model that possess different attack properties, without acquiring new training data or changing model architecture. The model owner can deploy one version at a time and replace a leaked version immediately with a new version. The newly deployed model version can resist adversarial attacks generated leveraging white-box access to one or all previously leaked versions. We show theoretically that this can be accomplished by incorporating parameterized hidden distributions into the model training data, forcing the model to learn task-irrelevant features uniquely defined by the chosen data. Additionally, optimal choices of hidden distributions can produce a sequence of model versions capable of resisting compound transferability attacks over time. Leveraging our analytical insights, we design and implement a practical model versioning method for DNN classifiers, which leads to significant robustness improvements over existing methods. We believe our work presents a promising direction for safeguarding DNN services beyond their initial deployment.
A Multi-Agent Security Testbed for the Analysis of Attacks and Defenses in Collaborative Sensor Fusion
Hallyburton, R. Spencer, Hunt, David, Luo, Shaocheng, Pajic, Miroslav
The performance and safety of autonomous vehicles (AVs) deteriorates under adverse environments and adversarial actors. The investment in multi-sensor, multi-agent (MSMA) AVs is meant to promote improved efficiency of travel and mitigate safety risks. Unfortunately, minimal investment has been made to develop security-aware MSMA sensor fusion pipelines leaving them vulnerable to adversaries. To advance security analysis of AVs, we develop the Multi-Agent Security Testbed, MAST, in the Robot Operating System (ROS2). Our framework is scalable for general AV scenarios and is integrated with recent multi-agent datasets. We construct the first bridge between AVstack and ROS and develop automated AV pipeline builds to enable rapid AV prototyping. We tackle the challenge of deploying variable numbers of agent/adversary nodes at launch-time with dynamic topic remapping. Using this testbed, we motivate the need for security-aware AV architectures by exposing the vulnerability of centralized multi-agent fusion pipelines to (un)coordinated adversary models in case studies and Monte Carlo analysis.
The landscape of Collective Awareness in multi-robot systems
Fernandez-Cortizas, Miguel, Perez-Saura, David, Sanz, Ricardo, Molina, Martin, Campoy, Pascual
The development of collective-aware multi-robot systems is crucial for enhancing the efficiency and robustness of robotic applications in multiple fields. These systems enable collaboration, coordination, and resource sharing among robots, leading to improved scalability, adaptability to dynamic environments, and increased overall system robustness. In this work, we want to provide a brief overview of this research topic and identify open challenges.
Cross-lingual Offensive Language Detection: A Systematic Review of Datasets, Transfer Approaches and Challenges
The growing prevalence and rapid evolution of offensive language in social media amplify the complexities of detection, particularly highlighting the challenges in identifying such content across diverse languages. This survey presents a systematic and comprehensive exploration of Cross-Lingual Transfer Learning (CLTL) techniques in offensive language detection in social media. Our study stands as the first holistic overview to focus exclusively on the cross-lingual scenario in this domain. We analyse 67 relevant papers and categorise these studies across various dimensions, including the characteristics of multilingual datasets used, the cross-lingual resources employed, and the specific CLTL strategies implemented. According to "what to transfer", we also summarise three main CLTL transfer approaches: instance, feature, and parameter transfer. Additionally, we shed light on the current challenges and future research opportunities in this field. Furthermore, we have made our survey resources available online, including two comprehensive tables that provide accessible references to the multilingual datasets and CLTL methods used in the reviewed literature.
ADCNet: a unified framework for predicting the activity of antibody-drug conjugates
Chen, Liye, Li, Biaoshun, Chen, Yihao, Lin, Mujie, Zhang, Shipeng, Li, Chenxin, Pang, Yu, Wang, Ling
Antibody-drug conjugate (ADC) has revolutionized the field of cancer treatment in the era of precision medicine due to their ability to precisely target cancer cells and release highly effective drug. Nevertheless, the realization of rational design of ADC is very difficult because the relationship between their structures and activities is difficult to understand. In the present study, we introduce a unified deep learning framework called ADCNet to help design potential ADCs. The ADCNet highly integrates the protein representation learning language model ESM-2 and small-molecule representation learning language model FG-BERT models to achieve activity prediction through learning meaningful features from antigen and antibody protein sequences of ADC, SMILES strings of linker and payload, and drug-antibody ratio (DAR) value. Based on a carefully designed and manually tailored ADC data set, extensive evaluation results reveal that ADCNet performs best on the test set compared to baseline machine learning models across all evaluation metrics. For example, it achieves an average prediction accuracy of 87.12%, a balanced accuracy of 0.8689, and an area under receiver operating characteristic curve of 0.9293 on the test set. In addition, cross-validation, ablation experiments, and external independent testing results further prove the stability, advancement, and robustness of the ADCNet architecture. For the convenience of the community, we develop the first online platform (https://ADCNet.idruglab.cn) for the prediction of ADCs activity based on the optimal ADCNet model, and the source code is publicly available at https://github.com/idrugLab/ADCNet.
QAnswer: Towards Question Answering Search over Websites
Guo, Kunpeng, Defretiere, Clement, Diefenbach, Dennis, Gravier, Christophe, Gourru, Antoine
Question Answering (QA) is increasingly used by search engines to provide results to their end-users, yet very few websites currently use QA technologies for their search functionality. To illustrate the potential of QA technologies for the website search practitioner, we demonstrate web searches that combine QA over knowledge graphs and QA over free text -- each being usually tackled separately. We also discuss the different benefits and drawbacks of both approaches for web site searches. We use the case studies made of websites hosted by the Wikimedia Foundation (namely Wikipedia and Wikidata). Differently from a search engine (e.g. Google, Bing, etc), the data are indexed integrally, i.e. we do not index only a subset, and they are indexed exclusively, i.e. we index only data available on the corresponding website.
What makes for a 'good' social actor? Using respect as a lens to evaluate interactions with language agents
Alberts, Lize, Keeling, Geoff, McCroskery, Amanda
With the growing popularity of dialogue agents based on large language models (LLMs), urgent attention has been drawn to finding ways to ensure their behaviour is ethical and appropriate. These are largely interpreted in terms of the 'HHH' criteria: making outputs more helpful and honest, and avoiding harmful (biased, toxic, or inaccurate) statements. Whilst this semantic focus is useful from the perspective of viewing LLM agents as mere mediums for information, it fails to account for pragmatic factors that can make the same utterance seem more or less offensive or tactless in different social situations. We propose an approach to ethics that is more centred on relational and situational factors, exploring what it means for a system, as a social actor, to treat an individual respectfully in a (series of) interaction(s). Our work anticipates a set of largely unexplored risks at the level of situated interaction, and offers practical suggestions to help LLM technologies behave as 'good' social actors and treat people respectfully.
GPT in Sheep's Clothing: The Risk of Customized GPTs
Antebi, Sagiv, Azulay, Noam, Habler, Edan, Ganon, Ben, Shabtai, Asaf, Elovici, Yuval
Generative artificial intelligence (GenAI) models are a type of deep learning neural network model capable of learning from large datasets and generating new content from a given context. They represent a significant leap in the ability of the artificial intelligence (AI) field to not just interpret data but also to create something new, including text, images, videos, code, and sound [2]. Large language models (LLMs) are a type of GenAI model designed to understand and generate natural language. The market for LLMs is estimated to reach 40.8 billion USD by 2029, up from 10.5 billion USD in 2022 [10]. Organizations are currently competing to develop the most sophisticated LLM capable of mimicking human-like conversations and tasks. This has led to the creation of models such as OpenAI's ChatGPT,
Robot Tape Manipulation for 3D Printing
Tushar, Nahid, Wu, Rencheng, She, Yu, Zhou, Wenchao, Shou, Wan
Progress has been made to innovate printing materials and printing processes, in terms of building blocks, joining mechanisms, forms of control, and transformation methods. Typically, material forms for 3D printing include solid filaments, wires, liquid resins, powders, and sheets (1). These feedstocks are transformed into discrete building units (such as droplets and lines) and placed, deposited, or solidified at designated locations for layer-by-layer manufacturing. However, 3D printing of continuous and flexible tape (with the geometric form in between filaments and sheets) without breaking or transformation remains underexplored and challenging. In the composite manufacturing industry, carbon fiber prepreg tapes are widely used for placement, which is called automated tape placement/laying (ATP/ATL) (3). Such ATP systems generally use heat and pressure to consolidate the composite materials (4, 5). However, ATP/ATL systems are typically mounted with large-scale gantry systems or robotic arms (4, 6-8). Such approaches require high capital investment and complex heavy equipment, which is not easily accessible to general researchers and difficult to integrate with desktop-scale 3D printing technologies.
Fine-grained Hallucination Detection and Editing for Language Models
Mishra, Abhika, Asai, Akari, Balachandran, Vidhisha, Wang, Yizhong, Neubig, Graham, Tsvetkov, Yulia, Hajishirzi, Hannaneh
Large language models (LMs) are prone to generate diverse factually incorrect statements, which are widely called hallucinations. Current approaches predominantly focus on coarse-grained automatic hallucination detection or editing, overlooking nuanced error levels. In this paper, we propose a novel task -- automatic fine-grained hallucination detection -- and present a comprehensive taxonomy encompassing six hierarchically defined types of hallucination. To facilitate evaluation, we introduce a new benchmark that includes fine-grained human judgments on two LM outputs across various domains. Our analysis reveals that ChatGPT and Llama 2-Chat exhibit hallucinations in 60% and 75% of their outputs, respectively, and a majority of these hallucinations fall into categories that have been underexplored. As an initial step to address this, we train FAVA, a retrieval-augmented LM by carefully designing synthetic data generations to detect and correct fine-grained hallucinations. On our benchmark, our automatic and human evaluations show that FAVA significantly outperforms ChatGPT on fine-grained hallucination detection by a large margin though a large room for future improvement still exists. FAVA's suggested edits also improve the factuality of LM-generated text, resulting in 5-10% FActScore improvements.