Many aspects of the design of efficient crowdsourcing processes, such as defining workers bonuses, fair prices and time limits of the tasks, involve knowledge of the likely duration of the task at hand. In this work we introduce a new timesensitive Bayesian aggregation method that simultaneously estimates a tasks duration and obtains reliable aggregations of crowdsourced judgments. Our method, called BCCTime, uses latent variables to represent the uncertainty about the workers completion time, the tasks duration and the workers accuracy. To relate the quality of a judgment to the time a worker spends on a task, our model assumes that each task is completed within a latent time window within which all workers with a propensity to genuinely attempt the labelling task (i.e., no spammers) are expected to submit their judgments. In contrast, workers with a lower propensity to valid labelling, such as spammers, bots or lazy labellers, are assumed to perform tasks considerably faster or slower than the time required by normal workers. Specifically, we use efficient message-passing Bayesian inference to learn approximate posterior probabilities of (i) the confusion matrix of each worker, (ii) the propensity to valid labelling of each worker, (iii) the unbiased duration of each task and (iv) the true label of each task. Using two real- world public datasets for entity linking tasks, we show that BCCTime produces up to 11% more accurate classifications and up to 100% more informative estimates of a tasks duration compared to stateoftheart methods.
Truong, Ngoc Cuong (University of Southampton) | McInerney, James (University of Southampton) | Tran-Thanh, Long (University of Southampton) | Costanza, Enrico (University of Southampton) | Ramchurn, Sarvapali (University of Southampton)
We address the problem of forecasting the usage of multiple electrical appliances by domestic users, with the aim of providing suggestions about the best time to run appliances in order to reduce carbon emissions and save money (assuming time-of-use pricing), while minimising the impact on the users' daily habits.An important challenge related to this problem is the modelling the everyday routine of the consumers and of the inter-dependencies between the use of different appliances. Given this, we develop an important building block of future home energy management systems: a prediction algorithm, based on a graphical model, that captures the everyday habits and the inter-dependency between appliances by exploiting their periodic features.We demonstrate through extensive empirical evaluations on real-world data from a prominent database that our approach outperforms existing methods by up to 47%.
We propose a method to determine the credibility of messages that are posted in participatory media (such as blogs), of use in recommender systems designed to provide users with messages that are considered to be the most credible to them. Our approach draws from theories developed in sociology, political science, and information science - this results in a method for evaluating the credibility of messages that is user-specific and sensitive to the social network in which the user resides. Our methodology rests on Bayesian learning, integrating new concepts of context and completeness of messages inspired by the strength of weak ties hypothesis from social network theory. We show that our credibility evaluation model can be used to significantly enhance the performance of collaborative filtering recommendation. Experimental validation is done using a dataset obtained from digg.com, a knowledge sharing website where users indicate their satisfaction with messages that are provided to them. Our results reinforce the value of using sociological insights in recommender system design.