Goto

Collaborating Authors

 Oceania


MetaAug: Meta-Data Augmentation for Post-Training Quantization

arXiv.org Artificial Intelligence

Post-Training Quantization (PTQ) has received significant attention because it requires only a small set of calibration data to quantize a full-precision model, which is more practical in real-world applications in which full access to a large training set is not available. However, it often leads to overfitting on the small calibration dataset. Several methods have been proposed to address this issue, yet they still rely on only the calibration set for the quantization and they do not validate the quantized model due to the lack of a validation set. In this work, we propose a novel meta-learning based approach to enhance the performance of post-training quantization. Specifically, to mitigate the overfitting problem, instead of only training the quantized model using the original calibration set without any validation during the learning process as in previous PTQ works, in our approach, we both train and validate the quantized model using two different sets of images. In particular, we propose a meta-learning based approach to jointly optimize a transformation network and a quantized model through bi-level optimization. The transformation network modifies the original calibration data and the modified data will be used as the training set to learn the quantized model with the objective that the quantized model achieves a good performance on the original calibration data. Extensive experiments on the widely used ImageNet dataset with different neural network architectures demonstrate that our approach outperforms the state-of-the-art PTQ methods. Code is available at this https URL.


Positive Text Reframing under Multi-strategy Optimization

arXiv.org Artificial Intelligence

Differing from sentiment transfer, positive reframing seeks to substitute negative perspectives with positive expressions while preserving the original meaning. With the emergence of pre-trained language models (PLMs), it is possible to achieve acceptable results by fine-tuning PLMs. Nevertheless, generating fluent, diverse and task-constrained reframing text remains a significant challenge. To tackle this issue, a \textbf{m}ulti-\textbf{s}trategy \textbf{o}ptimization \textbf{f}ramework (MSOF) is proposed in this paper. Starting from the objective of positive reframing, we first design positive sentiment reward and content preservation reward to encourage the model to transform the negative expressions of the original text while ensuring the integrity and consistency of the semantics. Then, different decoding optimization approaches are introduced to improve the quality of text generation. Finally, based on the modeling formula of positive reframing, we propose a multi-dimensional re-ranking method that further selects candidate sentences from three dimensions: strategy consistency, text similarity and fluency. Extensive experiments on two Seq2Seq PLMs, BART and T5, demonstrate our framework achieves significant improvements on unconstrained and controlled positive reframing tasks.


EaTVul: ChatGPT-based Evasion Attack Against Software Vulnerability Detection

arXiv.org Artificial Intelligence

Recently, deep learning has demonstrated promising results in enhancing the accuracy of vulnerability detection and identifying vulnerabilities in software. However, these techniques are still vulnerable to attacks. Adversarial examples can exploit vulnerabilities within deep neural networks, posing a significant threat to system security. This study showcases the susceptibility of deep learning models to adversarial attacks, which can achieve 100% attack success rate (refer to Table 5). The proposed method, EaTVul, encompasses six stages: identification of important samples using support vector machines, identification of important features using the attention mechanism, generation of adversarial data based on these features using ChatGPT, preparation of an adversarial attack pool, selection of seed data using a fuzzy genetic algorithm, and the execution of an evasion attack. Extensive experiments demonstrate the effectiveness of EaTVul, achieving an attack success rate of more than 83% when the snippet size is greater than 2. Furthermore, in most cases with a snippet size of 4, EaTVul achieves a 100% attack success rate. The findings of this research emphasize the necessity of robust defenses against adversarial attacks in software vulnerability detection.


Faster Image2Video Generation: A Closer Look at CLIP Image Embedding's Impact on Spatio-Temporal Cross-Attentions

arXiv.org Artificial Intelligence

This paper investigates the role of CLIP image embeddings within the Stable Video Diffusion (SVD) framework, focusing on their impact on video generation quality and computational efficiency. Our findings indicate that CLIP embeddings, while crucial for aesthetic quality, do not significantly contribute towards the subject and background consistency of video outputs. Moreover, the computationally expensive cross-attention mechanism can be effectively replaced by a simpler linear layer. This layer is computed only once at the first diffusion inference step, and its output is then cached and reused throughout the inference process, thereby enhancing efficiency while maintaining high-quality outputs. Building on these insights, we introduce the VCUT, a training-free approach optimized for efficiency within the SVD architecture. VCUT eliminates temporal cross-attention and replaces spatial cross-attention with a one-time computed linear layer, significantly reducing computational load. The implementation of VCUT leads to a reduction of up to 322T Multiple-Accumulate Operations (MACs) per video and a decrease in model parameters by up to 50M, achieving a 20% reduction in latency compared to the baseline. Our approach demonstrates that conditioning during the Semantic Binding stage is sufficient, eliminating the need for continuous computation across all inference steps and setting a new standard for efficient video generation.


Harmfully Manipulated Images Matter in Multimodal Misinformation Detection

arXiv.org Artificial Intelligence

Nowadays, misinformation is widely spreading over various social media platforms and causes extremely negative impacts on society. To combat this issue, automatically identifying misinformation, especially those containing multimodal content, has attracted growing attention from the academic and industrial communities, and induced an active research topic named Multimodal Misinformation Detection (MMD). Typically, existing MMD methods capture the semantic correlation and inconsistency between multiple modalities, but neglect some potential clues in multimodal content. Recent studies suggest that manipulated traces of the images in articles are non-trivial clues for detecting misinformation. Meanwhile, we find that the underlying intentions behind the manipulation, e.g., harmful and harmless, also matter in MMD. Accordingly, in this work, we propose to detect misinformation by learning manipulation features that indicate whether the image has been manipulated, as well as intention features regarding the harmful and harmless intentions of the manipulation. Unfortunately, the manipulation and intention labels that make these features discriminative are unknown. To overcome the problem, we propose two weakly supervised signals as alternatives by introducing additional datasets on image manipulation detection and formulating two classification tasks as positive and unlabeled learning problems. Based on these ideas, we propose a novel MMD method, namely Harmfully Manipulated Images Matter in MMD (HAMI-M3D). Extensive experiments across three benchmark datasets can demonstrate that HAMI-M3D can consistently improve the performance of any MMD baselines.


On Behalf of the Stakeholders: Trends in NLP Model Interpretability in the Era of LLMs

arXiv.org Artificial Intelligence

Recent advancements in NLP systems, particularly with the introduction of LLMs, have led to widespread adoption of these systems by a broad spectrum of users across various domains, impacting decision-making, the job market, society, and scientific research. This surge in usage has led to an explosion in NLP model interpretability and analysis research, accompanied by numerous technical surveys. Yet, these surveys often overlook the needs and perspectives of explanation stakeholders. In this paper, we address three fundamental questions: Why do we need interpretability, what are we interpreting, and how? By exploring these questions, we examine existing interpretability paradigms, their properties, and their relevance to different stakeholders. We further explore the practical implications of these paradigms by analyzing trends from the past decade across multiple research fields. To this end, we retrieved thousands of papers and employed an LLM to characterize them. Our analysis reveals significant disparities between NLP developers and non-developer users, as well as between research fields, underscoring the diverse needs of stakeholders. For example, explanations of internal model components are rarely used outside the NLP field. We hope this paper informs the future design, development, and application of methods that align with the objectives and requirements of various stakeholders.


IBMEA: Exploring Variational Information Bottleneck for Multi-modal Entity Alignment

arXiv.org Artificial Intelligence

Multi-modal entity alignment (MMEA) aims to identify equivalent entities between multi-modal knowledge graphs (MMKGs), where the entities can be associated with related images. Most existing studies integrate multi-modal information heavily relying on the automatically-learned fusion module, rarely suppressing the redundant information for MMEA explicitly. To this end, we explore variational information bottleneck for multi-modal entity alignment (IBMEA), which emphasizes the alignment-relevant information and suppresses the alignment-irrelevant information in generating entity representations. Specifically, we devise multi-modal variational encoders to generate modal-specific entity representations as probability distributions. Then, we propose four modal-specific information bottleneck regularizers, limiting the misleading clues in refining modal-specific entity representations. Finally, we propose a modal-hybrid information contrastive regularizer to integrate all the refined modal-specific representations, enhancing the entity similarity between MMKGs to achieve MMEA. We conduct extensive experiments on two cross-KG and three bilingual MMEA datasets. Experimental results demonstrate that our model consistently outperforms previous state-of-the-art methods, and also shows promising and robust performance in low-resource and high-noise data scenarios.


Canada used drones before and Tokyo gold could be 'tarnished'

BBC News

Canada national team officials have used drones prior to the Paris Olympics and their Tokyo 2020 women's gold medal could be tarnished, officials said on Friday. The developments emerged after Bev Priestman was removed as Olympics head coach for Canada's women's team, following the flying of a drone over New Zealand's training session on Monday. Priestman, 38, was judged as "highly likely" to have been aware of the incident, leading to her suspension by Canada Soccer. Canadian media reported that both of the country's senior teams - men's and women's - have relied on drones for years. Canada Soccer chief executive Kevin Blue confirmed he had received "anecdotal feedback" related to drone use during the men's team's run to the Copa America semi-finals this summer and that coach Jesse Marsch had only been made aware of it after the event.


Canadian Olympic Committee says spying scandal 'could tarnish' women's Tokyo gold medal

FOX News

The drone scandal surrounding the Canadian women's soccer team could have bigger implications than just this year's Games in Paris. Head coach Bev Priestman was removed from her position on Thursday night after two staff members were sent home from Paris after an investigation found that analyst Joseph Lombardi had used a drone to spy on New Zealand's practice sessions. Head coach Beverly Priestman reacts during the Women's Gold Medal match between Canada and Sweden on day 14 of the Tokyo 2020 Olympic Games at International Stadium Yokohama on Aug. 6, 2021 in Yokohama, Kanagawa, Japan. "Over the past 24 hours, additional information has come to our attention regarding previous drone use against opponents, predating the Paris 2024 Olympic Games," Canada Soccer CEO Kevin Blue said in a statement. "In light of these new revelations, Canada Soccer has made the decision to suspend Women's National Soccer Team Head Coach, Bev Priestman for the remainder of the Paris 2024 Olympic Games, and until the completion of our recently announced independent external review."


Priestman removed as Olympic boss over drone incident

BBC News

Canada women's football manager Beverly Priestman has been removed as Olympic head coach and suspended by the country's football federation as the fall out continued after a drone was flown over New Zealand's training session on Monday. Canada Soccer said it took the action because "over the past 24 hours, additional information has come to our attention regarding previous drone use against opponents, predating the Paris 2024 Olympic Games". English-born Priestman, 38, had "voluntarily" withdrew from her side's opening 2-0 victory over the Kiwis on Thursday, while Jasmine Mander, Priestman's assistant, was sent home along with "unaccredited analyst" Joseph Lombardi. On Thursday a French court said Lombardi had been handed an eight-month suspended jail sentence after pleading guilty to flying a drone in an urban area without a licence. In a statement Canada Soccer chief executive Kevin Blue confirmed Priestman will be suspended for the remainder of the Games while an "independent external review" takes place.