Goto

Collaborating Authors

 Bucharest


Watermarking Decision Tree Ensembles

arXiv.org Artificial Intelligence

Protecting the intellectual property of machine learning models is a hot topic and many watermarking schemes for deep neural networks have been proposed in the literature. Unfortunately, prior work largely neglected the investigation of watermarking techniques for other types of models, including decision tree ensembles, which are a state-of-the-art model for classification tasks on non-perceptual data. In this paper, we present the first watermarking scheme designed for decision tree ensembles, focusing in particular on random forest models. We discuss watermark creation and verification, presenting a thorough security analysis with respect to possible attacks. We finally perform an experimental evaluation of the proposed scheme, showing excellent results in terms of accuracy and security against the most relevant threats.


RoQLlama: A Lightweight Romanian Adapted Language Model

arXiv.org Artificial Intelligence

The remarkable achievements obtained by open-source large language models (LLMs) in recent years have predominantly been concentrated on tasks involving the English language. In this paper, we aim to advance the performance of Llama2 models on Romanian tasks. We tackle the problem of reduced computing resources by using QLoRA for training. We release RoQLlama-7b, a quantized LLM, which shows equal or improved results compared to its full-sized counterpart when tested on seven Romanian downstream tasks in the zero-shot setup. Also, it consistently achieves higher average scores across all few-shot prompts. Additionally, we introduce a novel Romanian dataset, namely RoMedQA, which contains single-choice medical questions in Romanian.


Neuron-Level Sequential Editing for Large Language Models

arXiv.org Artificial Intelligence

This work explores sequential model editing in large language models (LLMs), a critical task that involves modifying internal knowledge within LLMs continuously through multi-round editing, each incorporating updates or corrections to adjust the model outputs without the need for costly retraining. Existing model editing methods, especially those that alter model parameters, typically focus on single-round editing and often face significant challenges in sequential model editing-most notably issues of model forgetting and failure. To address these challenges, we introduce a new model editing method, namely \textbf{N}euron-level \textbf{S}equential \textbf{E}diting (NSE), tailored for supporting sequential model editing. Specifically, we optimize the target layer's hidden states using the model's original weights to prevent model failure. Furthermore, we iteratively select neurons in multiple layers for editing based on their activation values to mitigate model forgetting. Our empirical experiments demonstrate that NSE significantly outperforms current modifying parameters model editing methods, marking a substantial advancement in the field of sequential model editing. Our code is released on \url{https://github.com/jianghoucheng/NSE}.


How Hard is this Test Set? NLI Characterization by Exploiting Training Dynamics

arXiv.org Artificial Intelligence

Natural Language Inference (NLI) evaluation is crucial for assessing language understanding models; however, popular datasets suffer from systematic spurious correlations that artificially inflate actual model performance. To address this, we propose a method for the automated creation of a challenging test set without relying on the manual construction of artificial and unrealistic examples. We categorize the test set of popular NLI datasets into three difficulty levels by leveraging methods that exploit training dynamics. This categorization significantly reduces spurious correlation measures, with examples labeled as having the highest difficulty showing markedly decreased performance and encompassing more realistic and diverse linguistic phenomena. When our characterization method is applied to the training set, models trained with only a fraction of the data achieve comparable performance to those trained on the full dataset, surpassing other dataset characterization techniques. Our research addresses limitations in NLI dataset construction, providing a more authentic evaluation of model performance with implications for diverse NLU applications.


MOSEL: 950,000 Hours of Speech Data for Open-Source Speech Foundation Model Training on EU Languages

arXiv.org Artificial Intelligence

The rise of foundation models (FMs), coupled with regulatory efforts addressing their risks and impacts, has sparked significant interest in open-source models. However, existing speech FMs (SFMs) fall short of full compliance with the open-source principles, even if claimed otherwise, as no existing SFM has model weights, code, and training data publicly available under open-source terms. In this work, we take the first step toward filling this gap by focusing on the 24 official languages of the European Union (EU). We collect suitable training data by surveying automatic speech recognition datasets and unlabeled speech corpora under open-source compliant licenses, for a total of 950k hours. Additionally, we release automatic transcripts for 441k hours of unlabeled data under the permissive CC-BY license, thereby facilitating the creation of open-source SFMs for the EU languages.


Enhancing Romanian Offensive Language Detection through Knowledge Distillation, Multi-Task Learning, and Data Augmentation

arXiv.org Artificial Intelligence

This paper highlights the significance of natural language processing (NLP) within artificial intelligence, underscoring its pivotal role in comprehending and modeling human language. Recent advancements in NLP, particularly in conversational bots, have garnered substantial attention and adoption among developers. This paper explores advanced methodologies for attaining smaller and more efficient NLP models. Specifically, we employ three key approaches: (1) training a Transformer-based neural network to detect offensive language, (2) employing data augmentation and knowledge distillation techniques to increase performance, and (3) incorporating multi-task learning with knowledge distillation and teacher annealing using diverse datasets to enhance efficiency. The culmination of these methods has yielded demonstrably improved outcomes.


Neural Contrast: Leveraging Generative Editing for Graphic Design Recommendations

arXiv.org Artificial Intelligence

Creating visually appealing composites requires optimizing both text and background for compatibility. Previous methods have focused on simple design strategies, such as changing text color or adding background shapes for contrast. These approaches are often destructive, altering text color or partially obstructing the background image. Another method involves placing design elements in non-salient and contrasting regions, but this isn't always effective, especially with patterned backgrounds. To address these challenges, we propose a generative approach using a diffusion model. This method ensures the altered regions beneath design assets exhibit low saliency while enhancing contrast, thereby improving the visibility of the design asset.


NER-Luxury: Named entity recognition for the fashion and luxury domain

arXiv.org Artificial Intelligence

From artistry to political economy, philosophers of Ancient Greece already discussed the meanings and ramifications of the idea of luxury (Berry, 1994). Over the last several decades, the luxury industry has morphed into a global market, one of the most valuable sectors in France, and an important sector in Europe. Nevertheless, based on aesthetic values of artistic directors, this sector has been difficult to map network effects, to quantify relevant signals, and understand optimal strategies. For many years, economists, theorists and scholars have been passionate about the pricing of luxury goods based on scarcity (Smith, 1776), on the mechanism of value according to wealthy buyers (Ricardo, 1817) (Marshall, 1890), on the social aspect of consuming luxury goods (Veblen, 1899), and on the psychological effects such as the scarcity principle, formalized in the "Commodity theory" (Brock, 1968). The economic theory of "Design Innovation and Fashion cycles" (Pesendorfer, 1995) and the response "Fashion Cycles in Economics" (Coelho et al., 2004) brings those observations to the economic field by quantifying the complex buyer interactions and the importance of branding, over the quality of raw materials, or craftsmanship. Similarly, in the socioeconomic sphere, Jean Baudrillard explained that in postindustrial societies "Sign value" (Baudrillard, 1968) has surpassed the other economic values based on production cost, and pure market value. To understand the value of luxury goods from a consumer perspective in 2024, "the Distinction" (Bourdieu, 1979), the sociology research on the cartography of social structure to understand logic of taste are no longer relevant due to the complexity of modern consumer paths, with the power of network effects with social media platforms (Rohlfs, 1974), the digital identity at the age of hyperreality (Baurdillard, 1981), and the luxury goods, as an asset class for investment strategy.


A Machine Learning-Driven Wireless System for Structural Health Monitoring

arXiv.org Artificial Intelligence

The paper presents a wireless system integrated with a machine learning (ML) model for structural health monitoring (SHM) of carbon fiber reinforced polymer (CFRP) structures, primarily targeting aerospace applications. The system collects data via carbon nanotube (CNT) piezoresistive sensors embedded within CFRP coupons, wirelessly transmitting these data to a central server for processing. A deep neural network (DNN) model predicts mechanical properties and can be extended to forecast structural failures, facilitating proactive maintenance and enhancing safety. The modular design supports scalability and can be embedded within digital twin frameworks, offering significant benefits to aircraft operators and manufacturers. The system utilizes an ML model with a mean absolute error (MAE) of 0.14 on test data for forecasting mechanical properties. Data transmission latency throughout the entire system is less than one second in a LAN setup, highlighting its potential for real-time monitoring applications in aerospace and other industries. However, while the system shows promise, challenges such as sensor reliability under extreme environmental conditions and the need for advanced ML models to handle diverse data streams have been identified as areas for future research.


DeCLIP: Decoding CLIP representations for deepfake localization

arXiv.org Artificial Intelligence

Generative models can create entirely new images, but they can also partially modify real images in ways that are undetectable to the human eye. In this paper, we address the challenge of automatically detecting such local manipulations. One of the most pressing problems in deepfake detection remains the ability of models to generalize to different classes of generators. In the case of fully manipulated images, representations extracted from large self-supervised models (such as CLIP) provide a promising direction towards more robust detectors. Here, we introduce DeCLIP, a first attempt to leverage such large pretrained features for detecting local manipulations. We show that, when combined with a reasonably large convolutional decoder, pretrained self-supervised representations are able to perform localization and improve generalization capabilities over existing methods. Unlike previous work, our approach is able to perform localization on the challenging case of latent diffusion models, where the entire image is affected by the fingerprint of the generator. Moreover, we observe that this type of data, which combines local semantic information with a global fingerprint, provides more stable generalization than other categories of generative methods.