The typical model development lifecycle consists of four phases: 1) problem scoping, 2) data definition and collection, 3) model training and iterative improvement through error analysis, and 4) model deployment in production and implementation of continuous monitoring and retraining [1]. While the first three phases are typically performed in an offline setting, model deployment represents the critical step where the ML model becomes available in a production environment, a live application, where it needs to process live data and ideally sustain performance over time to keep delivering value. Model monitoring refers to the process of evaluating the quality of the production data and the performance of the model according to relevant metrics over time. When either data quality or model performance does not meet predefined criteria, a monitoring warning can be triggered, to alert the model owners. Defining an effective model monitoring and retraining strategy is key to successful ML model deployment since it can safeguard model quality over prolonged periods of time.

AI-assisted nuclei segmentation in histopathological images is a crucial task in the diagnosis and treatment of cancer diseases. It decreases the time required to manually screen microscopic tissue images and can resolve the conflict between pathologists during diagnosis. Deep Learning has proven useful in such a task. However, lack of labeled data is a significant barrier for deep learning-based approaches. In this study, we propose a novel approach to nuclei segmentation that leverages the available labelled and unlabelled data. The proposed method combines the strengths of both transductive and inductive learning, which have been previously attempted separately, into a single framework. Inductive learning aims at approximating the general function and generalizing to unseen test data, while transductive learning has the potential of leveraging the unlabelled test data to improve the classification. To the best of our knowledge, this is the first study to propose such a hybrid approach for medical image segmentation. Moreover, we propose a novel two-stage transductive inference scheme. We evaluate our approach on MoNuSeg benchmark to demonstrate the efficacy and potential of our method.

Artificial neural networks typically struggle in generalizing to out-of-context examples. One reason for this limitation is caused by having datasets that incorporate only partial information regarding the potential correlational structure of the world. In this work, we propose TIDA (Targeted Image-editing Data Augmentation), a targeted data augmentation method focused on improving models' human-like abilities (e.g., gender recognition) by filling the correlational structure gap using a text-to-image generative model. More specifically, TIDA identifies specific skills in captions describing images (e.g., the presence of a specific gender in the image), changes the caption (e.g., "woman" to "man"), and then uses a text-to-image model to edit the image in order to match the novel caption (e.g., uniquely changing a woman to a man while maintaining the context identical). Based on the Flickr30K benchmark, we show that, compared with the original data set, a TIDA-enhanced dataset related to gender, color, and counting abilities induces better performance in several image captioning metrics. Furthermore, on top of relying on the classical BLEU metric, we conduct a fine-grained analysis of the improvements of our models against the baseline in different ways. We compared text-to-image generative models and found different behaviors of the image captioning models in terms of encoding visual encoding and textual decoding.

Text-based game environments are challenging because agents must deal with long sequences of text, execute compositional actions using text and learn from sparse rewards. We address these challenges by proposing Language Decision Transformers (LDTs), a framework that is based on transformer language models and decision transformers (DTs). Our LDTs extend DTs with 3 components: (1) exponential tilt to guide the agent towards high obtainable goals, (2) novel goal conditioning methods yielding better results than the traditional return-to-go (sum of all future rewards), and (3) a model of future observations that improves agent performance. LDTs are the first to address offline RL with DTs on these challenging games. Our experiments show that LDTs achieve the highest scores among many different types of agents on some of the most challenging Jericho games, such as Enchanter.

High dimension, low sample size (HDLSS) problems are numerous among real-world applications of machine learning. From medical images to text processing, traditional machine learning algorithms are usually unsuccessful in learning the best possible concept from such data. In a previous work, we proposed a dissimilarity-based approach for multi-view classification, the Random Forest Dissimilarity (RFD), that perfoms state-of-the-art results for such problems. In this work, we transpose the core principle of this approach to solving HDLSS classification problems, by using the RF similarity measure as a learned precomputed SVM kernel (RFSVM). We show that such a learned similarity measure is particularly suited and accurate for this classification context. Experiments conducted on 40 public HDLSS classification datasets, supported by rigorous statistical analyses, show that the RFSVM method outperforms existing methods for the majority of HDLSS problems and remains at the same time very competitive for low or non-HDLSS problems.

Microsoft's AI ambitions are moving a bit backwards: Today, the company has confirmed that it's bringing Copilot AI to Windows 10. At first, it'll be available to Windows Insider users in an upcoming Release Preview update, where Copilot will appear on the right side of the Task Bar. Once selected, you'll see the familiar Copilot interface, which you can use to ask the AI questions, manage Windows features or interact with documents. Microsoft says the Copilot window won't overlap with desktop content or block open windows. If this all sounds familiar, it's because Windows Central reported that Windows 10 would be getting Copilot earlier this month.

Understanding visually situated language requires recognizing text and visual elements, and interpreting complex layouts. State-of-the-art methods commonly use specialized pre-processing tools, such as optical character recognition (OCR) systems, that map document image inputs to extracted information in the space of textual tokens, and sometimes also employ large language models (LLMs) to reason in text token space. However, the gains from external tools and LLMs come at the cost of increased computational and engineering complexity. In this paper, we ask whether small pretrained image-to-text models can learn selective text or layout recognition and reasoning as an intermediate inference step in an end-to-end model for pixel-level visual language understanding. We incorporate the outputs of such OCR tools, LLMs, and larger multimodal models as intermediate ``rationales'' on training data, and train a small student model to predict both rationales and answers for input questions based on those training examples. A student model based on Pix2Struct (282M parameters) achieves consistent improvements on three visual document understanding benchmarks representing infographics, scanned documents, and figures, with improvements of more than 4\% absolute over a comparable Pix2Struct model that predicts answers directly.

Ensuring a neural network is not relying on protected attributes (e.g., race, sex, age) for predictions is crucial in advancing fair and trustworthy AI. While several promising methods for removing attribute bias in neural networks have been proposed, their limitations remain under-explored. In this work, we mathematically and empirically reveal an important limitation of attribute bias removal methods in presence of strong bias. Specifically, we derive a general non-vacuous information-theoretical upper bound on the performance of any attribute bias removal method in terms of the bias strength. We provide extensive experiments on synthetic, image, and census datasets to verify the theoretical bound and its consequences in practice. Our findings show that existing attribute bias removal methods are effective only when the inherent bias in the dataset is relatively weak, thus cautioning against the use of these methods in smaller datasets where strong attribute bias can occur, and advocating the need for methods that can overcome this limitation.

Deep Neural Networks (DNNs) have demonstrated remarkable capabilities in addressing intricate tasks like image classification, object detection, speech recognition, natural language processing, and document analysis, at times even surpassing human performance [21,23,24]. This success has ignited a surge in exploring the viability of DNNs across diverse real-world domains, including biometric authentication, mobile facial recognition for security, and malware detection. However, given the sensitive nature of the data in these critical applications, incorporating safety, security, and robust verification into their design has become paramount. However, studies have revealed that even slight modifications in input data can effectively mislead cutting-edge, well-trained networks, causing inaccuracies in their predictions [12, 32, 40]. The arena of network verification has primarily concentrated on image inputs, particularly emphasizing the assurance of safety and robustness in various classification neural networks [2, 7, 19, 31, 43, 44]. Previous investigations have scrutinized a range of network architectures, encompassing feed-forward neural networks (FFNNs [42]), convolutional neural networks (CNNs [44]), semantic segmentation networks (SSNs [43]), and a few using Recurrent Neural Networks (RNNs [41]) employing diverse set-based reachability tools such as Neural Network Verification (NNV [26,45]) and JuliaReach [6], among others. Models utilizing NNs for audio classification have found application in diverse tasks, ranging from Music Genre Classification [8, 10, 11] and Environmental Sound Classification [4, 9, 13] to Audio Generation [33, 36]. Therefore, formal verification of audio classification systems holds paramount importance in ensuring their reliability and safety, particularly in safety-critical applications such as autonomous vehicles [35, 46], medical diagnosis [15, 30], and industrial monitoring [47]. This study introduces an extension, building upon the foundations laid by two recent studies [34,41] in the domain of formal verification.

In the context of information systems, text sanitization techniques are used to identify and remove sensitive data to comply with security and regulatory requirements. Even though many methods for privacy preservation have been proposed, most of them are focused on the detection of entities from specific domains (e.g., credit card numbers, social security numbers), lacking generality and requiring customization for each desirable domain. Moreover, removing words is, in general, a drastic measure, as it can degrade text coherence and contextual information. Less severe measures include substituting a word for a safe alternative, yet it can be challenging to automatically find meaningful substitutions. We present a zero-shot text sanitization technique that detects and substitutes potentially sensitive information using Large Language Models. Our evaluation shows that our method excels at protecting privacy while maintaining text coherence and contextual information, preserving data utility for downstream tasks.