Collaborating Authors

Basis refinement strategies for linear value function approximation in MDPs

Neural Information Processing Systems

We provide a theoretical framework for analyzing basis function construction for linear value function approximation in Markov Decision Processes (MDPs). We show that important existing methods, such as Krylov bases and Bellman-error-based methods are a special case of the general framework we develop. We provide a general algorithmic framework for computing basis function refinements which "respect" the dynamics of the environment, and we derive approximation error bounds that apply for any algorithm respecting this general framework. We also show how, using ideas related to bisimulation metrics, one can translate basis refinement into a process of finding "prototypes" that are diverse enough to represent the given MDP. Papers published at the Neural Information Processing Systems Conference.

Representation Discovery in Sequential Decision Making

AAAI Conferences

Automatically constructing novel representations of tasks from analysis of state spaces is a longstanding fundamental challenge in AI. I review recent progress on this problem for sequential decision making tasks modeled as Markov decision processes. Specifically, I discuss three classes of representation discovery problems: finding functional, state, and temporal abstractions. I describe solution techniques varying along several dimensions: diagonalization or dilation methods using approximate or exact transition models; reward-specific vs reward-invariant methods; global vs. local representation construction methods; multiscale vs. flat discovery methods; and finally, orthogonal vs. redundant representa- tion discovery methods. I conclude by describing a number of open problems for future work.

Efficient Graph-based Word Sense Induction


The paper was first presented at TextGraphs-2018, a workshop series at The 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) on June 6, 2018 in New Orleans. This new approach to word-sense induction comes from the work of the Lexalytics Magic Machines AI Labs, launched in 2017 in partnership with the University of Massachusetts Amherst's Center for Data Science and Northwestern University's Medill School of Journalism, Media and Integrated Marketing Communications to drive innovation in AI. Word sense induction (WSI) is a challenging task of natural language processing whose goal is to categorize and identify multiple senses of polysemous words from raw text without the help of predefined sense inventory like WordNet (Miller, 1995). The problem is sometimes also called unsupervised word sense disambiguation (Agirre et al., 2006; Pelevina et al., 2016). An effective WSI has wide applications.

Polynomial Interpolation


Polynomial interpolation is a fundamental problem in numerical analysis. Given a set of points in the plane, its objective is to find the smallest-degree polynomial that passes through these points. In this article, we'll explore two methods of addressing this problem: the Vandermonde matrix method, and the Lagrange polynomial method. Let's consider the following set of points: Since we have 4 points in the dataset, we'll attempt to find a third-degree polynomial that passes through them: For the determinant to be nonzero, all the x-coordinates of the data points must be unique. As we saw previously, the Vandermonde matrix method requires finding the inverse of an n by n matrix, n being the number of points, which is not efficient for large datasets.

Color Opponency Constitutes a Sparse Representation for the Chromatic Structure of Natural Scenes

Neural Information Processing Systems

The human visual system encodes the chromatic signals conveyed by the three types of retinal cone photoreceptors in an opponent fashion. This color opponency has been shown to constitute an efficient encoding by spectral decorrelation of the receptor signals. We analyze the spatial and chromatic structure of natural scenes by decomposing the spectral images into a set of linear basis functions such that they constitute a representation with minimal redundancy. Independent component analysis finds the basis functions that transforms the spatiochromatic data such that the outputs (activations) are statistically as independent as possible, i.e. least redundant. The resulting basis functions show strong opponency along an achromatic direction (luminance edges), along a blueyellow direction, and along a red-blue direction.