Overview
FSMR: A Feature Swapping Multi-modal Reasoning Approach with Joint Textual and Visual Clues
Li, Shuang, Wang, Jiahua, Wen, Lijie
Multi-modal reasoning plays a vital role in bridging the gap between textual and visual information, enabling a deeper understanding of the context. This paper presents the Feature Swapping Multi-modal Reasoning (FSMR) model, designed to enhance multi-modal reasoning through feature swapping. FSMR leverages a pre-trained visual-language model as an encoder, accommodating both text and image inputs for effective feature representation from both modalities. It introduces a unique feature swapping module, enabling the exchange of features between identified objects in images and corresponding vocabulary words in text, thereby enhancing the model's comprehension of the interplay between images and text. To further bolster its multi-modal alignment capabilities, FSMR incorporates a multi-modal cross-attention mechanism, facilitating the joint modeling of textual and visual information. During training, we employ image-text matching and cross-entropy losses to ensure semantic consistency between visual and language elements. Extensive experiments on the PMR dataset demonstrate FSMR's superiority over state-of-the-art baseline models across various performance metrics.
A Survey of using Large Language Models for Generating Infrastructure as Code
Srivatsa, Kalahasti Ganesh, Mukhopadhyay, Sabyasachi, Katrapati, Ganesh, Shrivastava, Manish
Infrastructure as Code (IaC) is a revolutionary approach which has gained significant prominence in the Industry. IaC manages and provisions IT infrastructure using machine-readable code by enabling automation, consistency across the environments, reproducibility, version control, error reduction and enhancement in scalability. However, IaC orchestration is often a painstaking effort which requires specialised skills as well as a lot of manual effort. Automation of IaC is a necessity in the present conditions of the Industry and in this survey, we study the feasibility of applying Large Language Models (LLM) to address this problem. LLMs are large neural network-based models which have demonstrated significant language processing abilities and shown to be capable of following a range of instructions within a broad scope. Recently, they have also been adapted for code understanding and generation tasks successfully, which makes them a promising choice for the automatic generation of IaC configurations. In this survey, we delve into the details of IaC, usage of IaC in different platforms, their challenges, LLMs in terms of code-generation aspects and the importance of LLMs in IaC along with our own experiments. Finally, we conclude by presenting the challenges in this area and highlighting the scope for future research.
Chinese Offensive Language Detection:Current Status and Future Directions
Xiao, Yunze, Bouamor, Houda, Zaghouani, Wajdi
Despite the considerable efforts being made to monitor and regulate user-generated content on social media platforms, the pervasiveness of offensive language, such as hate speech or cyberbullying, in the digital space remains a significant challenge. Given the importance of maintaining a civilized and respectful online environment, there is an urgent and growing need for automatic systems capable of detecting offensive speech in real time. However, developing effective systems for processing languages such as Chinese presents a significant challenge, owing to the language's complex and nuanced nature, which makes it difficult to process automatically. This paper provides a comprehensive overview of offensive language detection in Chinese, examining current benchmarks and approaches and highlighting specific models and tools for addressing the unique challenges of detecting offensive language in this complex language. The primary objective of this survey is to explore the existing techniques and identify potential avenues for further research that can address the cultural and linguistic complexities of Chinese.
Fully Zeroth-Order Bilevel Programming via Gaussian Smoothing
Aghasi, Alireza, Ghadimi, Saeed
We are particularly interested in the setting where neither ex plicit knowledge about f,g are available nor their unbiased stochastic derivatives. In this zeroth-order setting, we assume that only noisy evaluations of f and g are available upon query to an oracle. The BLP problem was first introduced by Bracken and McGill in t he 1970s [7] followed by a more general form of problem involving joint constraints of outer and inner variables. This is a fundamental problem in engineering and economics with dire ct applications in problems such as decision making [48], supply chain [61, 59], network design [51, 43], transportation and planning [16, 83], and optimal design [4, 32]. More recently, BLP has f ound applications in many areas of machine learning and artificial intelligence. Zeroth-order methods apply to many optimization problems ( including the BLP) where for various reasons such as complexity, lack of access to an accurat e model, or computational limitations, there is no or limited access to the objective gradient.
NLP for Counterspeech against Hate: A Survey and How-To Guide
Bonaldi, Helena, Chung, Yi-Ling, Abercrombie, Gavin, Guerini, Marco
In recent years, counterspeech has emerged as one of the most promising strategies to fight online hate. These non-escalatory responses tackle online abuse while preserving the freedom of speech of the users, and can have a tangible impact in reducing online and offline violence. Recently, there has been growing interest from the Natural Language Processing (NLP) community in addressing the challenges of analysing, collecting, classifying, and automatically generating counterspeech, to reduce the huge burden of manually producing it. In particular, researchers have taken different directions in addressing these challenges, thus providing a variety of related tasks and resources. In this paper, we provide a guide for doing research on counterspeech, by describing - with detailed examples - the steps to undertake, and providing best practices that can be learnt from the NLP studies on this topic. Finally, we discuss open challenges and future directions of counterspeech research in NLP.
Homomorphic WiSARDs: Efficient Weightless Neural Network training over encrypted data
Neumann, Leonardo, Guimarães, Antonio, Aranha, Diego F., Borin, Edson
The widespread application of machine learning algorithms is a matter of increasing concern for the data privacy research community, and many have sought to develop privacy-preserving techniques for it. Among existing approaches, the homomorphic evaluation of ML algorithms stands out by performing operations directly over encrypted data, enabling strong guarantees of confidentiality. The homomorphic evaluation of inference algorithms is practical even for relatively deep Convolution Neural Networks (CNNs). However, training is still a major challenge, with current solutions often resorting to lightweight algorithms that can be unfit for solving more complex problems, such as image recognition. This work introduces the homomorphic evaluation of Wilkie, Stonham, and Aleksander's Recognition Device (WiSARD) and subsequent Weightless Neural Networks (WNNs) for training and inference on encrypted data. Compared to CNNs, WNNs offer better performance with a relatively small accuracy drop. We develop a complete framework for it, including several building blocks that can be of independent interest. Our framework achieves 91.7% accuracy on the MNIST dataset after only 3.5 minutes of encrypted training (multi-threaded), going up to 93.8% in 3.5 hours. For the HAM10000 dataset, we achieve 67.9% accuracy in just 1.5 minutes, going up to 69.9% after 1 hour. Compared to the state of the art on the HE evaluation of CNN training, Glyph (Lou et al., NeurIPS 2020), these results represent a speedup of up to 1200 times with an accuracy loss of at most 5.4%. For HAM10000, we even achieved a 0.65% accuracy improvement while being 60 times faster than Glyph. We also provide solutions for small-scale encrypted training. In a single thread on a desktop machine using less than 200MB of memory, we train over 1000 MNIST images in 12 minutes or over the entire Wisconsin Breast Cancer dataset in just 11 seconds.
Bayesian Nonparametrics: An Alternative to Deep Learning
Bayesian nonparametric models offer a flexible and powerful framework for statistical model selection, enabling the adaptation of model complexity to the intricacies of diverse datasets. This survey intends to delve into the significance of Bayesian nonparametrics, particularly in addressing complex challenges across various domains such as statistics, computer science, and electrical engineering. By elucidating the basic properties and theoretical foundations of these nonparametric models, this survey aims to provide a comprehensive understanding of Bayesian nonparametrics and their relevance in addressing complex problems, particularly in the domain of multi-object tracking. Through this exploration, we uncover the versatility and efficacy of Bayesian nonparametric methodologies, paving the way for innovative solutions to intricate challenges across diverse disciplines.
Expectations Versus Reality: Evaluating Intrusion Detection Systems in Practice
Hesford, Jake, Cheng, Daniel, Wan, Alan, Huynh, Larry, Kim, Seungho, Kim, Hyoungshick, Hong, Jin B.
However, it is or flows. Where a dataset does not contain both of these also a challenge when trying to compare them and choose the formats, adapting it into the form expected by a given IDS is best one for your needs, because there is no standardisation non-trivial, where the expected format is not the one provided due to the complexity of the environment that these IDSs by the dataset authors. This discrepancy presents challenges were designed for. In order to determine to what degree in obtaining satisfactory results when an IDS and dataset are IDSs can be adapted to different environments, we compare incompatible without significant processing [1]. Our evaluation their performance across common Network Intrusion process was further complicated by the necessity of converting Detection Systems (NIDS) datasets. This approach aims to these datasets into formats compatible with various IDS provide a more standardized basis for comparison, taking into solutions. This data wrangling could amplify the errors and account different variables such as attack types, networking inconsistencies inherent in the datasets.
The State of Lithium-Ion Battery Health Prognostics in the CPS Era
Shinde, Gaurav, Mohapatra, Rohan, Krishan, Pooja, Garg, Harish, Prabhu, Srikanth, Das, Sanchari, Masum, Mohammad, Sengupta, Saptarshi
Lithium-ion batteries (Li-ion) have revolutionized energy storage technology, becoming integral to our daily lives by powering a diverse range of devices and applications. Their high energy density, fast power response, recyclability, and mobility advantages have made them the preferred choice for numerous sectors. This paper explores the seamless integration of Prognostics and Health Management within batteries, presenting a multidisciplinary approach that enhances the reliability, safety, and performance of these powerhouses. Remaining useful life (RUL), a critical concept in prognostics, is examined in depth, emphasizing its role in predicting component failure before it occurs. The paper reviews various RUL prediction methods, from traditional models to cutting-edge data-driven techniques. Furthermore, it highlights the paradigm shift toward deep learning architectures within the field of Li-ion battery health prognostics, elucidating the pivotal role of deep learning in addressing battery system complexities. Practical applications of PHM across industries are also explored, offering readers insights into real-world implementations.This paper serves as a comprehensive guide, catering to both researchers and practitioners in the field of Li-ion battery PHM.
Enhancing Efficiency in Vision Transformer Networks: Design Techniques and Insights
Heidari, Moein, Azad, Reza, Kolahi, Sina Ghorbani, Arimond, René, Niggemeier, Leon, Sulaiman, Alaa, Bozorgpour, Afshin, Aghdam, Ehsan Khodapanah, Kazerouni, Amirhossein, Hacihaliloglu, Ilker, Merhof, Dorit
Intrigued by the inherent ability of the human visual system to identify salient regions in complex scenes, attention mechanisms have been seamlessly integrated into various Computer Vision (CV) tasks. Building upon this paradigm, Vision Transformer (ViT) networks exploit attention mechanisms for improved efficiency. This review navigates the landscape of redesigned attention mechanisms within ViTs, aiming to enhance their performance. This paper provides a comprehensive exploration of techniques and insights for designing attention mechanisms, systematically reviewing recent literature in the field of CV. This survey begins with an introduction to the theoretical foundations and fundamental concepts underlying attention mechanisms. We then present a systematic taxonomy of various attention mechanisms within ViTs, employing redesigned approaches. A multi-perspective categorization is proposed based on their application, objectives, and the type of attention applied. The analysis includes an exploration of the novelty, strengths, weaknesses, and an in-depth evaluation of the different proposed strategies. This culminates in the development of taxonomies that highlight key properties and contributions. Finally, we gather the reviewed studies along with their available open-source implementations at our \href{https://github.com/mindflow-institue/Awesome-Attention-Mechanism-in-Medical-Imaging}{GitHub}\footnote{\url{https://github.com/xmindflow/Awesome-Attention-Mechanism-in-Medical-Imaging}}. We aim to regularly update it with the most recent relevant papers.