We address the problem of active logistic regression in the realizable setting. It is well known that active learning can require exponentially fewer label queries compared to passive learning, in some cases using $\log \frac{1}{\eps}$ rather than $\poly(1/\eps)$ labels to get error $\eps$ larger than the optimum. We present the first algorithm that is polynomially competitive with the optimal algorithm on every input instance, up to factors polylogarithmic in the error and domain size. In particular, if any algorithm achieves label complexity polylogarithmic in $\eps$, so does ours. Our algorithm is based on efficient sampling and can be extended to learn more general class of functions. We further support our theoretical results with experiments demonstrating performance gains for logistic regression compared to existing active learning algorithms.

Linear representation hypothesis posits that high-level concepts are encoded as linear directions in the representation spaces of LLMs. Park et al. (2024) formalize this notion by unifying multiple interpretations of linear representation, such as 1-dimensional subspace representation and interventions, using a causal inner product. However, their framework relies on single-token counterfactual pairs and cannot handle ambiguous contrasting pairs, limiting its applicability to complex or context-dependent concepts. We introduce a new notion of binary concepts as unit vectors in a canonical representation space, and utilize LLMs' (neural) activation differences along with maximum likelihood estimation (MLE) to compute concept directions (i.e., steering vectors). Our method, Sum of Activation-base Normalized Difference (SAND), formalizes the use of activation differences modeled as samples from a von Mises-Fisher (vMF) distribution, providing a principled approach to derive concept directions. We extend the applicability of Park et al. (2024) by eliminating the dependency on unembedding representations and single-token pairs. Through experiments with LLaMA models across diverse concepts and benchmarks, we demonstrate that our lightweight approach offers greater flexibility, superior performance in activation engineering tasks like monitoring and manipulation.

We study how to automatically select and adapt multiple abstractions or representations of the world to support model-based reinforcement learning. We address the challenges of transfer learning in heterogeneous environments with varying tasks. We present an efficient, online framework that, through a sequence of tasks, learns a set of relevant representations to be used in future tasks. Without pre-defined mapping strategies, we introduce a general approach to support transfer learning across different state spaces. We demonstrate the potential impact of our system through improved jumpstart and faster convergence to near optimum policy in two benchmark domains.

This article describes Compaq QUICKSOURCE, an electronic problem-solving and information system for Compaq's line of networked printers. A major goal in designing this system was to empower Compaq's customers with expert system technology, allowing them to solve advanced network printer problems entirely on their own. In its first-generation system, SMART, the objective was to provide expert knowledge to Compaq's help-desk operation to better and more quickly answer customer calls and problems. Because the product would be used by a diverse and heterogeneous set of users, a significant amount of human factors research and analysis was performed as part of system design and implementation.

This article describes Compaq QUICKSOURCE, an electronic problem-solving and information system for Compaq's line of networked printers. A major goal in designing this system was to empower Compaq's customers with expert system technology, allowing them to solve advanced network printer problems entirely on their own. This process minimizes customer down time; reduces the number of telephone calls to the Compaq Customer-Support Center (resulting in monetary savings); improves customer satisfaction; and, perhaps most importantly, differentiates Compaq printers in the market-place by providing the best and most technologically advanced customer-support facility. This approach also represents a reengineering of Compaq's customer-support strategy and implementation. In its first-generation system, SMART, the objective was to provide expert knowledge to Compaq's help-desk operation to better and more quickly answer customer calls and problems. QUICKSOURCE is a second-generation system in that the customer-support function is put directly in the hands of the consumers (an example of knowledge publishing). As a result, its design presented a number of different and challenging issues. Because the product would be used by a diverse and heterogeneous set of users, a significant amount of human factors research and analysis was performed as part of system design and implementation. The analysis also dictated certain decisions about the organization and design of the expert system component. Since September 1992, Compaq has shipped more than 3000 copies of QUICKSOURCE.