Adapting state-of-the-art Large Language Models (LLMs) like GPT-4 and Gemini for specific tasks is challenging. Due to the opacity in their parameters, embeddings, and even output probabilities, existing fine-tuning adaptation methods are inapplicable. Consequently, adapting these black-box LLMs is only possible through their API services, raising concerns about transparency, privacy, and cost. To address these challenges, we introduce BBox-Adapter, a novel lightweight adapter for black-box LLMs. BBox-Adapter distinguishes target and source domain data by treating target data as positive and source data as negative. It employs a ranking-based Noise Contrastive Estimation (NCE) loss to promote the likelihood of target domain data while penalizing that of the source domain. Furthermore, it features an online adaptation mechanism, which incorporates real-time positive data sampling from ground-truth, human, or AI feedback, coupled with negative data from previous adaptations. Extensive experiments demonstrate BBox-Adapter's effectiveness and cost efficiency. It improves model performance by up to 6.77% across diverse tasks and domains, while reducing training and inference costs by 31.30x and 1.84x, respectively.

This paper explores the potential for leveraging Large Language Models (LLM) in the realm of online advertising systems. We delve into essential requirements including privacy, latency, reliability as well as the satisfaction of users and advertisers which such a system must fulfill. We further introduce a general framework for LLM advertisement, consisting of modification, bidding, prediction, and auction modules. Different design considerations for each module is presented, with an in-depth examination of their practicality and the technical challenges inherent to their implementation. Finally, we explore the prospect of LLM-based dynamic creative optimization as a means to significantly enhance the appeal of advertisements to users and discuss its additional challenges.

Causal graph discovery is a significant problem with applications across various disciplines. However, with observational data alone, the underlying causal graph can only be recovered up to its Markov equivalence class, and further assumptions or interventions are necessary to narrow down the true graph. This work addresses the causal discovery problem under the setting of stochastic interventions with the natural goal of minimizing the number of interventions performed. We propose the following stochastic intervention model which subsumes existing adaptive noiseless interventions in the literature while capturing scenarios such as fat-hand interventions and CRISPR gene knockouts: any intervention attempt results in an actual intervention on a random subset of vertices, drawn from a distribution dependent on attempted action. Under this model, we study the two fundamental problems in causal discovery of verification and search and provide approximation algorithms with polylogarithmic competitive ratios and provide some preliminary experimental results.

Research into large-scale crop monitoring has flourished due to increased accessibility to satellite imagery. This review delves into previously unexplored and under-explored areas in sugarcane health monitoring and disease/pest detection using satellite-based spectroscopy and Machine Learning (ML). It discusses key considerations in system development, including relevant satellites, vegetation indices, ML methods, factors influencing sugarcane reflectance, optimal growth conditions, common diseases, and traditional detection methods. Many studies highlight how factors like crop age, soil type, viewing angle, water content, recent weather patterns, and sugarcane variety can impact spectral reflectance, affecting the accuracy of health assessments via spectroscopy. However, these variables have not been fully considered in the literature. In addition, the current literature lacks comprehensive comparisons between ML techniques and vegetation indices. We address these gaps in this review. We discuss that, while current findings suggest the potential for an ML-driven satellite spectroscopy system for monitoring sugarcane health, further research is essential. This paper offers a comprehensive analysis of previous research to aid in unlocking this potential and advancing the development of an effective sugarcane health monitoring system using satellite technology.

We consider contextual bandits with graph feedback, a class of interactive learning problems with richer structures than vanilla contextual bandits, where taking an action reveals the rewards for all neighboring actions in the feedback graph under all contexts. Unlike the multi-armed bandits setting where a growing literature has painted a near-complete understanding of graph feedback, much remains unexplored in the contextual bandits counterpart. In this paper, we make inroads into this inquiry by establishing a regret lower bound $\Omega(\sqrt{\beta_M(G) T})$, where $M$ is the number of contexts, $G$ is the feedback graph, and $\beta_M(G)$ is our proposed graph-theoretical quantity that characterizes the fundamental learning limit for this class of problems. Interestingly, $\beta_M(G)$ interpolates between $\alpha(G)$ (the independence number of the graph) and $\mathsf{m}(G)$ (the maximum acyclic subgraph (MAS) number of the graph) as the number of contexts $M$ varies. We also provide algorithms that achieve near-optimal regrets for important classes of context sequences and/or feedback graphs, such as transitively closed graphs that find applications in auctions and inventory control. In particular, with many contexts, our results show that the MAS number completely characterizes the statistical complexity for contextual bandits, as opposed to the independence number in multi-armed bandits.

The class imbalance problem is a classic classification problem, which arises because the number of negative samples (i.e., majority class) in the data set is much larger than the number of positive samples (i.e., minority class)[4]. This type of problem is common in many fields. For example, in the field of financial fraud, the occurrence of occasional small-probability fraud will cause huge economic losses. Therefore, accurately identifying positive samples will be the key to the class imbalance problem. The first difficulty in the class imbalance problem is mainly due to the rarity of positive samples, which has two connotations[2]: One is absolutely rare, which makes the data not representative enough and has a lot of noise; the other is relatively rare, which causes the feature space to overlap seriously, making it hard for the model to accurately separate the two classes. The second reason is the potential discrimination toward positive samples by current mainstream classifiers. Many current models treat the majority and minority classes equally when evaluating classification accuracy, resulting in the direction of model evaluation being naturally biased towards the majorities; the third reason is the potential discrimination toward important samples in positive samples by the oversampling model. SMOTE, as a classic oversampling method to solve class imbalance[1], only selects the data randomly when expanding the minorities, which may result in more serious feature space overlap because of the ignoration of important samples in minorities. To solve the various problems mentioned above, we propose a hierarchical Smote Based on Marginalized Minorities(MM-SMOTE). First, we use the basic SVM classifier to roughly classify the data, and obtain the support vectors in minorities as important samples for sampling; then assign weights to those support vectors based on their distance to the decision hyperplane; and then based on the k-nearest neighbors of support vectors, we used an adaptive oversampling to generate synthetic samples; finally, synthetic samples are used to augment the original kernel function of the basic SVM to form a new classifier.

Scripted agents have predominantly won the five previous iterations of the IEEE microRTS (µRTS) competitions hosted at CIG and CoG. Despite Deep Reinforcement Learning (DRL) algorithms making significant strides in real-time strategy (RTS) games, their adoption in this primarily academic competition has been limited due to the considerable training resources required and the complexity inherent in creating and debugging such agents. In a benchmark without performance constraints, RAISocketAI regularly defeated the two prior competition winners. This first competition-winning DRL submission can be a benchmark for future microRTS competitions and a starting point for future DRL research. Iteratively fine-tuning the base policy and transfer learning to specific maps were critical to RAISocketAI's winning performance. These strategies can be used to economically train future DRL agents. Further work in Imitation Learning using Behavior Cloning and fine-tuning these models with DRL has proven promising as an efficient way to bootstrap models with demonstrated, competitive behaviors. Deep reinforcement learning (DRL) has proven to be powerful at solving complex problems requiring several steps to achieve a goal, such as Atari games (Mnih et al., 2013), continuous control tasks (Lillicrap et al., 2016), and even real-time strategy (RTS) games like StarCraft II (Vinyals et al., 2019). The StarCraft II grandmaster agent AlphaStar was trained with thousands of CPUs and GPUs/TPUs for several weeks. RTS games are particularly challenging for DRL for several reasons: (1) the observation and action spaces are large and varied with different terrain and unit types; (2) each unit type can have different actions and abilities; (3) each action can control several units at once; (4) rewards are sparse (win, loss, or tie) and delayed by possibly several thousand timesteps; (5) winning requires combining tactical (micro) and strategic (macro) decisions; (6) actions must be taken in real-time (i.e., the game won't wait for the agent to take an action); (7) the agent might not have full visibility of the game state (i.e., fog of war); and (8) events in the game might be non-deterministic. It includes many aspects of RTS games, simplified: different unit types, unit-specific actions, terrain, resource collection and utilization to build units, and unit-to-unit combat where units have different strengths and weaknesses. The IEEE microRTS competitions have been hosted at the Conference on Games (CoG) nearly every year since 2019 and at the Conference on Computational Intelligence and Games (CIG) before that since 2017 (Ontañón et al., 2018).

Background: Machine learning (ML) methods often fail with data that deviates from their training distribution. This is a significant concern for ML-enabled devices in clinical settings, where data drift may cause unexpected performance that jeopardizes patient safety. Method: We propose a ML-enabled Statistical Process Control (SPC) framework for out-of-distribution (OOD) detection and drift monitoring. SPC is advantageous as it visually and statistically highlights deviations from the expected distribution. To demonstrate the utility of the proposed framework for monitoring data drift in radiological images, we investigated different design choices, including methods for extracting feature representations, drift quantification, and SPC parameter selection. Results: We demonstrate the effectiveness of our framework for two tasks: 1) differentiating axial vs. non-axial computed tomography (CT) images and 2) separating chest x-ray (CXR) from other modalities. For both tasks, we achieved high accuracy in detecting OOD inputs, with 0.913 in CT and 0.995 in CXR, and sensitivity of 0.980 in CT and 0.984 in CXR. Our framework was also adept at monitoring data streams and identifying the time a drift occurred. In a simulation with 100 daily CXR cases, we detected a drift in OOD input percentage from 0-1% to 3-5% within two days, maintaining a low false-positive rate. Through additional experimental results, we demonstrate the framework's data-agnostic nature and independence from the underlying model's structure. Conclusion: We propose a framework for OOD detection and drift monitoring that is agnostic to data, modality, and model. The framework is customizable and can be adapted for specific applications.

To find New Physics or to refine our knowledge of the Standard Model at the LHC is an enterprise that involves many factors. We focus on taking advantage of available information and pour our effort in re-thinking the usual data-driven ABCD method to improve it and to generalize it using Bayesian Machine Learning tools. We propose that a dataset consisting of a signal and many backgrounds is well described through a mixture model. Signal, backgrounds and their relative fractions in the sample can be well extracted by exploiting the prior knowledge and the dependence between the different observables at the event-by-event level with Bayesian tools. We show how, in contrast to the ABCD method, one can take advantage of understanding some properties of the different backgrounds and of having more than two independent observables to measure in each event. In addition, instead of regions defined through hard cuts, the Bayesian framework uses the information of continuous distribution to obtain soft-assignments of the events which are statistically more robust. To compare both methods we use a toy problem inspired by $pp\to hh\to b\bar b b \bar b$, selecting a reduced and simplified number of processes and analysing the flavor of the four jets and the invariant mass of the jet-pairs, modeled with simplified distributions. Taking advantage of all this information, and starting from a combination of biased and agnostic priors, leads us to a very good posterior once we use the Bayesian framework to exploit the data and the mutual information of the observables at the event-by-event level. We show how, in this simplified model, the Bayesian framework outperforms the ABCD method sensitivity in obtaining the signal fraction in scenarios with $1\%$ and $0.5\%$ true signal fractions in the dataset. We also show that the method is robust against the absence of signal.

The efficacy of language models, particularly in reasoning tasks, is significantly impacted by longer text lengths than those seen in training [19, 2, 15]. This phenomenon, referred to as "Length Generalization" or "Length Extrapolation" in the literature [25, 30], is also common in models based on the Transformer architecture [20, 16, 8, 32]. Notably, even Large Language Models (LLMs), known for their strong performance in a wide range of tasks and domains, are not immune to this problem [2, 5]. Recent research tried to address this challenge by modifications to the positional embeddings [25, 6, 7, 19, 13] or by using prompting strategies such as scratchpad [23] and chain-of-thought reasoning [28]. Nevertheless, there remains a lack of datasets specifically designed for the systematic evaluation of the problem.