Bayesian Learning
A Decision-Theoretic Academic Advisor
Guerin, Joshua T. (University of Kentucky)
Despite the importance of academic advising to student This paper describes work related to stochastic modeling and success, academic advising services are often found to decision-theoretic (DT) planning methods applicable to the be inconsistent, and are often not treated as a high priority real-world domain of academic advising.
Learning to Rank Under Multiple Annotators
Wu, Ou (NLPR, Institute of Automation, Chinese Academy of Sciences) | Hu, Weiming (NLPR, Institute of Automation, Chinese Academy of Sciences) | Gao, Jun (NLPR, Institute of Automation, Chinese Academy of Sciences)
Learning to rank has received great attention in recent years as it plays a crucial role in information retrieval. The existing concept of learning to rank assumes that each training sample is associated with an instance and a reliable label. However, in practice, this assumption does not necessarily hold true. This study focuses on the learning to rank when each training instance is labeled by multiple annotators that may be unreliable. In such a scenario, no accurate labels can be obtained. This study proposes two learning approaches. One is to simply estimate the ground truth first and then to learn a ranking model with it. The second approach is a maximum likelihood learning approach which estimates the ground truth and learns the ranking model iteratively. The two approaches have been tested on both synthetic and real-world data. The results reveal that the maximum likelihood approach outperforms the first approach significantly and is comparable of achieving results with the learning model considering reliable labels. Further more, both the approaches have been applied for ranking the Web visual clutter.
Agent-Oriented Incremental Team and Activity Recognition
Masato, Daniele (University of Aberdeen) | Norman, Timothy J. (University of Aberdeen) | Vasconcelos, Wamberto W. (University of Aberdeen) | Sycara, Katia (Carnegie Mellon University)
Monitoring team activity is beneficial when human teams cooperate in the enactment of a joint plan. Monitoring allows teams to maintain awareness of each other's progress within the plan and it enables anticipation of information needs. Humans find this difficult, particularly in time-stressed and uncertain environments. In this paper we introduce a probabilistic model, based on Conditional Random Fields, to automatically recognise the composition of teams and the team activities in relation to a plan. The team composition and activities are recognised incrementally by interpreting a stream of spatio-temporal observations.
Improving Performance of Topic Models by Variable Grouping
Bart, Evgeniy (Palo Alto Research Center)
Topic models have a wide range of applications, including modeling of text documents, images, user preferences, product rankings, and many others. However, learning optimal models may be difficult, especially for large problems. The reason is that inference techniques such as Gibbs sampling often converge to suboptimal models due to the abundance of local minima in large datasets. In this paper, we propose a general method of improving the performance of topic models. The method, called 'grouping transform', works by introducing auxiliary variables which represent assignments of the original model tokens to groups. Using these auxiliary variables, it becomes possible to resample an entire group of tokens at a time. This allows the sampler to make larger state space moves. As a result, better models are learned and performance is improved. The proposed ideas are illustrated on several topic models and several text and image datasets. We show that the grouping transform significantly improves performance over standard models.
Learning Optimal Bayesian Networks Using A* Search
Yuan, Changhe (Mississippi State University) | Malone, Brandon (Mississippi State University) | Wu, Xiaojian (University of Massachusetts)
This paper formulates learning optimal Bayesian network as a shortest path finding problem. An A* search algorithm is introduced to solve the problem. With the guidance of a consistent heuristic, the algorithm learns an optimal Bayesian networkby only searching the most promising parts of the solution space. Empirical results show that the A*search algorithm significantly improves the time and space efficiency of existing methods on a set of benchmark datasets.
Biclustering-Driven Ensemble of Bayesian Belief Network Classifiers for Underdetermined Problems
Pansombut, Tatdow (North Carolina State University, Oak Ridge National Laboratory) | Hendrix, William (North Carolina State University, Oak Ridge National Laboratory) | Gao, Zekai J. (Zhejiang University) | Harrison, Brent E. (North Carolina State University, Oak Ridge National Laboratory) | Samatova, Nagiza F. (North Carolina State University, Oak Ridge National Laboratory)
In this paper, we present BENCH (BiclusteringdrivenENsemble of Classifiers), an algorithm toconstruct an ensemble of classifiers through concurrentfeature and data point selection guided byunsupervised knowledge obtained from biclustering.BENCH is designed for underdeterminedproblems. In our experiments, we use Bayesian BeliefNetwork (BBN) classifiers as base classifiers inthe ensemble; however, BENCH can be applied toother classification models as well. We show thatBENCH is able to increase prediction accuracy ofa single classifier and traditional ensemble of classifiersby up to 15% on three microarray datasetsusing various weighting schemes for combining individualpredictions in the ensemble.
A Maximum Likelihood Approach Towards Aggregating Partial Orders
Xia, Lirong (Duke University) | Conitzer, Vincent (Duke University)
In many of the possible applications as well as the theoretical models of computational social choice,the agentsโ preferences are represented as partialorders. In this paper, we extend the maximum likelihood approach for defining โoptimalโ voting rules to this setting. We consider distributions in which the pairwise comparisons / incomparabilities between alternatives are drawn i.i.d. We call suchmodels pairwise-independentmodels and show that they correspond to a class of voting rules that we call pairwise scoring rules. This generalizes rulessuch as Kemeny and Borda. Moreover, we show that Borda is the only pairwise scoring rule that satisfies neutrality, when the outcome space is the set of all alternatives. We then study which voting rules defined for linear orders can be extended to partial orders via our MLE model. We show that any weakly neutral outcome scoring rule (includingany ranking/candidate scoring rule) based onthe weighted majority graph can be represented as the MLE of a weakly neutral pairwise-independent model. Therefore, all such rules admit natural extensionsto profiles of partial orders. Finally, we propose a specific MLE model ฯ k for generating a set of k winning alternatives, and study the computational complexity of winner determination for the MLE of ฯ k .
A General MCMC Method for Bayesian Inference in Logic-Based Probabilistic Modeling
Sato, Taisuke (Tokyo Institute of Technology)
We propose a general MCMC method for Bayesian inference in logic-based probabilistic modeling. It covers a broad class of generative models including Bayesian networks and PCFGs. The idea is to generalize an MCMC method for PCFGs to the one for a Turing-complete probabilistic modeling language PRISM in the context of statistical abduction where parse trees are replaced with explanations. We describe how to estimate the marginal probability of data from MCMC samples and how to perform Bayesian Viterbi inference using an example of Naive Bayes model augmented with a hidden variable.
Learning Decision Rules from Data Streams
Gama, Joรฃo (University of Porto) | Kosina, Petr (University of Porto)
However, it has been shown that the antecedents of individual rules Decision rules, which can provide good interpretability may contain irrelevant conditions. C4.5rules (Quinlan, 1993) and flexibility for data mining tasks, uses an optimization procedure to simplify conditions. The have received very little attention in the stream optimization is done in two phases. First, each rule is generalized mining community so far. In this work we introduce by deleting conditions that do not seem to be helpful a new algorithm to learn rule sets, designed in discriminating the classes. A greedy search method is for open-ended data streams.
Continuous Correlated Beta Processes
Goetschalckx, Robby (University of Dundee) | Poupart, Pascal (University of Waterloo) | Hoey, Jesse (University of Waterloo)
In this paper we consider a (possibly continuous) space of Bernoulli experiments. We assume that the Bernoulli distributions of the points are correlated. All evidence data comes in the form of successful or failed experiments at different points. Current state-of-the-art methods for expressing a distribution over a continuum of Bernoulli distributions use logistic Gaussian processes or Gaussian copula processes. However, both of these require computationally expensive matrix operations (cubic in the general case). We introduce a more intuitive approach, directly correlating beta distributions by sharing evidence between them according to a kernel function, an approach which has linear time complexity. The approach can easily be extended to multiple outcomes, giving a continuous correlated Dirichlet process.This approach can be used for classification (both binary and multi-class) and learning the actual probabilities of the Bernoulli distributions. We show results for a number of data sets, as well as a case-study where a mixture of continuous beta processes is used as part of an automated stroke rehabilitation system.