Hybrid-MST: A Hybrid Active Sampling Strategy for Pairwise Preference Aggregation

Neural Information Processing Systems

In this paper we present a hybrid active sampling strategy for pairwise preference aggregation, which aims at recovering the underlying rating of the test candidates from sparse and noisy pairwise labeling. Our method employs Bayesian optimization framework and Bradley-Terry model to construct the utility function, then to obtain the Expected Information Gain (EIG) of each pair. For computational efficiency, Gaussian-Hermite quadrature is used for estimation of EIG. In this work, a hybrid active sampling strategy is proposed, either using Global Maximum (GM) EIG sampling or Minimum Spanning Tree (MST) sampling in each trial, which is determined by the test budget. The proposed method has been validated on both simulated and real-world datasets, where it shows higher preference aggregation ability than the state-of-the-art methods.


Automated Multiagent Preference Aggregation Using Fuzzy Quantifiers

AAAI Conferences

Ronald R. Yager Machine Intelligence Institute Iona College New Rochelle, NY 10801 An issue that is of considerable interest in distributed systems is the selection of a solution acceptable to a collection agents when each agent has their own preferences among a set of alternative possible solutions. Recently Rosenchein and Zlotkin [1] have presented a comprehensive work discussing this issue. The process of finding such a solution can be seen as a kind of negotiation amongsthese agents. This class of problems have a long history in the economic literature. In the fuzzy set literature one finds a considerable body of literature devoted to the closely related problem of multi-criteria aggregation starting with the classic work of Bellman and Zadeh [2].


Preference Aggregation with Incomplete CP-Nets

AAAI Conferences

Generalized CP-nets (gCP-nets) extend standard CP-nets by allowing conditional preference tables to be incomplete. Such generality is desirable, as in practice users may want to express preferences over the values of a variable that depend only on partial assignments for other variables. In this paper we study aggregation of gCP-nets, under the name of multiple gCP-nets (mgCP-nets). Inspired by existing research on mCP-nets, we define different semantics for mgCP-nets and study the complexity of prominent reasoning tasks such as dominance, consistency and various notions of optimality.


DATELINE: Deep Plackett-Luce Model with Uncertainty Measurements

arXiv.org Machine Learning

The aggregation of k-ary preferences is a historical and important problem, since it has many real-world applications, such as peer grading, presidential elections and restaurant ranking. Meanwhile, variants of Plackett-Luce model has been applied to aggregate k-ary preferences. However, there are two urgent issues still existing in the current variants. First, most of them ignore feature information. Namely, they consider k-ary preferences instead of instance-dependent k-ary preferences. Second, these variants barely consider the uncertainty in k-ary preferences provided by agnostic crowds. In this paper, we propose Deep plAckeTt-luce modEL wIth uNcertainty mEasurements (DATELINE), which can address both issues simultaneously. To address the first issue, we employ deep neural networks mapping each instance into its ranking score in Plackett-Luce model. Then, we present a weighted Plackett-Luce model to solve the second issue, where the weight is a dynamic uncertainty vector measuring the worker quality. More importantly, we provide theoretical guarantees for DATELINE to justify its robustness.


Hybrid-MST: A Hybrid Active Sampling Strategy for Pairwise Preference Aggregation

Neural Information Processing Systems

In this paper we present a hybrid active sampling strategy for pairwise preference aggregation, which aims at recovering the underlying rating of the test candidates from sparse and noisy pairwise labelling. Our method employs Bayesian optimization framework and Bradley-Terry model to construct the utility function, then to obtain the Expected Information Gain (EIG) of each pair. For computational efficiency, Gaussian-Hermite quadrature is used for estimation of EIG. In this work, a hybrid active sampling strategy is proposed, either using Global Maximum (GM) EIG sampling or Minimum Spanning Tree (MST) sampling in each trial, which is determined by the test budget. The proposed method has been validated on both simulated and real-world datasets, where it shows higher preference aggregation ability than the state-of-the-art methods.