Key, Peter
Using Convolutional Neural Networks to Analyze Function Properties from Images
Lewenberg, Yoad (The Hebrew University of Jerusalem, Israel) | Bachrach, Yoram (Microsoft Research) | Kash, Ian (Microsoft Research) | Key, Peter (Microsoft Research)
We propose a system for determining properties of mathematical functions given an image of their graph representation. We demonstrate our approach for two-dimensional graphs (curves of single variable functions) and three-dimensional graphs (surfaces of two variable functions), studying the properties of convexity and symmetry. Our method uses a Convolutional Neural Network which classifies functions according to these properties, without using any hand-crafted features. We propose algorithms for randomly constructing functions with convexity or symmetry properties, and use the images generated by these algorithms to train our network. Our system achieves a high accuracy on this task, even for functions where humans find it difficult to determine the function's properties from its image.
Hotspotting โ A Probabilistic Graphical Model For Image Object Localization Through Crowdsourcing
Salek, Mahyar (Microsoft Research, Cambridge) | Bachrach, Yoram (Microsoft Research, Cambridge) | Key, Peter (Microsoft Research, Cambridge)
Object localization is an image annotation task which consists of finding the location of a target object in an image. It is common to crowdsource annotation tasks and aggregate responses to estimate the true annotation. While for other kinds of annotations consensus is simple and powerful, it cannot be applied to object localization as effectively due to the task's rich answer space and inherent noise in responses. We propose a probabilistic graphical model to localize objects in images based on responses from the crowd. We improve upon natural aggregation methods such as the mean and the median by simultaneously estimating the difficulty level of each question and skill level of every participant. We empirically evaluate our model on crowdsourced data and show that our method outperforms simple aggregators both in estimating the true locations and in ranking participants by their ability. We also propose a simple adaptive sourcing scheme that works well for very sparse datasets.
Congestion Games with Agent Failures
Meir, Reshef (Hebrew University and Microsoft Research, Herzlia) | Tennenholtz, Moshe (Technion-Israel Institute of Technology and Microsoft Research, Herzlia) | Bachrach, Yoram (Microsoft Research, Cambridge) | Key, Peter (Microsoft Research, Cambridge)
We propose a natural model for agent failures in congestion games. In our model, each of the agents may fail to participate in the game, introducing uncertainty regarding the set of active agents. We examine how such uncertainty may change the Nash equilibria (NE) of the game. We prove that although the perturbed game induced by the failure model is not always a congestion game, it still admits at least one pure Nash equilibrium. Then, we turn to examine the effect of failures on the maximal social cost in any NE of the perturbed game. We show that in the limit case where failure probability is negligible new equilibria never emerge, and that the social cost may decrease but it never increases. For the case of non-negligible failure probabilities, we provide a full characterization of the maximal impact of failures on the social cost under worst-case equilibrium outcomes.
Quality Expectation-Variance Tradeoffs in Crowdsourcing Contests
Gao, Xi Alice (Harvard University) | Bachrach, Yoram (Microsoft Research) | Key, Peter (Microsoft Research) | Graepel, Thore (Microsoft Research)
We examine designs for crowdsourcing contests, where participants compete for rewards given to superior solutions of a task. We theoretically analyze tradeoffs between the expectation and variance of the principal's utility (i.e. the best solution's quality), and empirically test our theoretical predictions using a controlled experiment on Amazon Mechanical Turk. Our evaluation method is also crowdsourcing based and relies on the peer prediction mechanism. Our theoretical analysis shows an expectation-variance tradeoff of the principal's utility in such contests through a Pareto efficient frontier. In particular, we show that the simple contest with 2 authors and the 2-pair contest have good theoretical properties. In contrast, our empirical results show that the 2-pair contest is the superior design among all designs tested, achieving the highest expectation and lowest variance of the principal's utility.