Goto

Collaborating Authors

 Gemmell, Jonathan


Evaluating Team Skill Aggregation in Online Competitive Games

arXiv.org Artificial Intelligence

One of the main goals of online competitive games is increasing player engagement by ensuring fair matches. These games use rating systems for creating balanced match-ups. Rating systems leverage statistical estimation to rate players' skills and use skill ratings to predict rank before matching players. Skill ratings of individual players can be aggregated to compute the skill level of a team. While research often aims to improve the accuracy of skill estimation and fairness of match-ups, less attention has been given to how the skill level of a team is calculated from the skill level of its members. In this paper, we propose two new aggregation methods and compare them with a standard approach extensively used in the research literature. We present an exhaustive analysis of the impact of these methods on the predictive performance of rating systems. We perform our experiments using three popular rating systems, Elo, Glicko, and TrueSkill, on three real-world datasets including over 100,000 battle royale and head-to-head matches. Our evaluations show the superiority of the MAX method over the other two methods in the majority of the tested cases, implying that the overall performance of a team is best determined by the performance of its most skilled member. The results of this study highlight the necessity of devising more elaborated methods for calculating a team's performance -- methods covering different aspects of players' behavior such as skills, strategy, or goals.


The Evaluation of Rating Systems in Team-based Battle Royale Games

arXiv.org Artificial Intelligence

Online competitive games have become a mainstream entertainment platform. To create a fair and exciting experience, these games use rating systems to match players with similar skills. While there has been an increasing amount of research on improving the performance of these systems, less attention has been paid to how their performance is evaluated. In this paper, we explore the utility of several metrics for evaluating three popular rating systems on a real-world dataset of over 25,000 team battle royale matches. Our results suggest considerable differences in their evaluation patterns. Some metrics were highly impacted by the inclusion of new players. Many could not capture the real differences between certain groups of players. Among all metrics studied, normalized discounted cumulative gain (NDCG) demonstrated more reliable performance and more flexibility. It alleviated most of the challenges faced by the other metrics while adding the freedom to adjust the focus of the evaluations on different groups of players.


Improved Results for Minimum Constraint Removal

AAAI Conferences

Given a set of obstacles and two designated points in the plane, the Minimum Constraint Removal problem asks for a minimum number of obstacles that can be removed so that a collision-free path exists between the two designated points. It is a well-studied problem in both robotic motion planning and wireless computing that has been shown to be NP-hard in various settings. In this work, we extend the study of Minimum Constraint Removal. We start by presenting refined NP-hardness reductions for the two cases: (1) when all the obstacles are axes-parallel rectangles, and (2) when all the obstacles are line segments such that no three intersect at the same point. These results improve on existing results in the literature. As a byproduct of our NP-hardness reductions, we prove that, unless the Exponential-Time Hypothesis (ETH) fails, Minimum Constraint Removal cannot be solved in subexponential time 2 o ( n ) , where n is the number of obstacles in the instance. This shows that significant improvement on the brute-force 2 O ( n ) -time algorithm is unlikely. We then present a subexponential-time algorithm for instances of Minimum Constraint Removal in which the number of obstacles that overlap at any point is constant; the algorithm runs in time 2 O (√ N ) , where N is the number of the vertices in the auxiliary graph associated with the instance of the problem. We show that significant improvement on this algorithm is unlikely by showing that, unless ETH fails, Minimum Constraint Removal with bounded overlap number cannot be solved in time 2 o (√ N ) . We describe several exact algorithms and approximation algorithms that leverage heuristics and discuss their performance in an extensive empirical simulation.


Recommendation in the Social Web

AI Magazine

Recommender systems are a means of personalizing the presentation of information to ensure that users see the items most relevant to them. The social web has added new dimensions to the way people interact on the Internet, placing the emphasis on user-generated content. Users in social networks create photos, videos and other artifacts, collaborate with other users, socialize with their friends and share their opinions online. This outpouring of material has brought increased attention to recommender systems, as a means of managing this vast universe of content. At the same time, the diversity and complexity of the data has meant new challenges for researchers in recommendation. This article describes the nature of recommendation research in social web applications and provides some illustrative examples of current research directions and techniques. It is difficult to overstate the impact of the social web. This new breed of social applications is reshaping nearly every human activity from the way people watch movies to how they overthrow governments. Facebook allows its members to maintain friendships whether they live next door or on another continent. With Twitter, users from celebrities to ordinary folks can launch their 140 character messages out to a diverse horde of ‘‘followers.” Flickr and YouTube users upload their personal media to share with the world, while Wikipedia editors collaborate on the world’s largest encyclopedia.