Goto

Collaborating Authors

 Uncertainty



Estimating Accuracy from Unlabeled Data: A Probabilistic Logic Approach

arXiv.org Machine Learning

We propose an efficient method to estimate the accuracy of classifiers using only unlabeled data. We consider a setting with multiple classification problems where the target classes may be tied together through logical constraints. For example, a set of classes may be mutually exclusive, meaning that a data instance can belong to at most one of them. The proposed method is based on the intuition that: (i) when classifiers agree, they are more likely to be correct, and (ii) when the classifiers make a prediction that violates the constraints, at least one classifier must be making an error. Experiments on four real-world data sets produce accuracy estimates within a few percent of the true accuracy, using solely unlabeled data. Our models also outperform existing state-of-the-art solutions in both estimating accuracies, and combining multiple classifier outputs. The results emphasize the utility of logical constraints in estimating accuracy, thus validating our intuition.


The Kernel Mixture Network: A Nonparametric Method for Conditional Density Estimation of Continuous Random Variables

arXiv.org Machine Learning

This paper introduces the kernel mixture network, a new method for nonparametric estimation of conditional probability densities using neural networks. We model arbitrarily complex conditional densities as linear combinations of a family of kernel functions centered at a subset of training points. The weights are determined by the outer layer of a deep neural network, trained by minimizing the negative log likelihood. This generalizes the popular quantized softmax approach, which can be seen as a kernel mixture network with square and non-overlapping kernels. We test the performance of our method on two important applications, namely Bayesian filtering and generative modeling. In the Bayesian filtering example, we show that the method can be used to filter complex nonlinear and non-Gaussian signals defined on manifolds. The resulting kernel mixture network filter outperforms both the quantized softmax filter and the extended Kalman filter in terms of model likelihood. Finally, our experiments on generative models show that, given the same architecture, the kernel mixture network leads to higher test set likelihood, less overfitting and more diversified and realistic generated samples than the quantized softmax approach.


CDS Rate Construction Methods by Machine Learning Techniques

arXiv.org Machine Learning

Regulators require financial institutions to estimate counterparty default risks from liquid CDS quotes for the valuation and risk management of OTC derivatives. However, the vast majority of counterparties do not have liquid CDS quotes and need proxy CDS rates. Existing methods cannot account for counterparty-specific default risks; we propose to construct proxy CDS rates by associating to illiquid counterparty liquid CDS Proxy based on Machine Learning Techniques. After testing 156 classifiers from 8 most popular classifier families, we found that some classifiers achieve highly satisfactory accuracy rates. Furthermore, we have rank-ordered the performances and investigated performance variations amongst and within the 8 classifier families. This paper is, to the best of our knowledge, the first systematic study of CDS Proxy construction by Machine Learning techniques, and the first systematic classifier comparison study based entirely on financial market data. Its findings both confirm and contrast existing classifier performance literature. Given the typically highly correlated nature of financial data, we investigated the impact of correlation on classifier performance. The techniques used in this paper should be of interest for financial institutions seeking a CDS Proxy method, and can serve for proxy construction for other financial variables. Some directions for future research are indicated.


Applying Bayes Theorem: Simulating the Monty Hall Problem with Python

#artificialintelligence

The Monty Hall problem was first featured on the classic game show "Let's make a Deal". In the final segment of the show, contestants were presented with a choice of three different doors. Behind two of the doors would be a goat, and behind the third would be an extravagant prize such as a car. The contestant begins the game by picking one door. The host, Monty Hall, would then open one of the remaining doors.


Evolving Ensemble Fuzzy Classifier

arXiv.org Artificial Intelligence

The concept of ensemble learning offers a promising avenue in learning from data streams under complex environments because it addresses the bias and variance dilemma better than its single model counterpart and features a reconfigurable structure, which is well suited to the given context. While various extensions of ensemble learning for mining non-stationary data streams can be found in the literature, most of them are crafted under a static base classifier and revisits preceding samples in the sliding window for a retraining step. This feature causes computationally prohibitive complexity and is not flexible enough to cope with rapidly changing environments. Their complexities are often demanding because it involves a large collection of offline classifiers due to the absence of structural complexities reduction mechanisms and lack of an online feature selection mechanism. A novel evolving ensemble classifier, namely Parsimonious Ensemble pENsemble, is proposed in this paper. pENsemble differs from existing architectures in the fact that it is built upon an evolving classifier from data streams, termed Parsimonious Classifier pClass. pENsemble is equipped by an ensemble pruning mechanism, which estimates a localized generalization error of a base classifier. A dynamic online feature selection scenario is integrated into the pENsemble. This method allows for dynamic selection and deselection of input features on the fly. pENsemble adopts a dynamic ensemble structure to output a final classification decision where it features a novel drift detection scenario to grow the ensemble structure. The efficacy of the pENsemble has been numerically demonstrated through rigorous numerical studies with dynamic and evolving data streams where it delivers the most encouraging performance in attaining a tradeoff between accuracy and complexity.


Joint Positioning and Radio Map Generation Based on Stochastic Variational Bayesian Inference for FWIPS

arXiv.org Machine Learning

Fingerprinting based WLAN indoor positioning system (FWIPS) provides a promising indoor positioning solution to meet the growing interests for indoor location-based services (e.g., indoor way finding or geo-fencing). FWIPS is preferred because it requires no additional infrastructure for deploying an FWIPS and achieving the position estimation by reusing the available WLAN and mobile devices, and capable of providing absolute position estimation. For fingerprinting based positioning (FbP), a model is created to provide reference values of observable features (e.g., signal strength from access point (AP)) as a function of location during offline stage. One widely applied method to build a complete and an accurate reference database (i.e. radio map (RM)) for FWIPS is carrying out a site survey throughout the region of interest (RoI). Along the site survey, the readings of received signal strength (RSS) from all visible APs at each reference point (RP) are collected. This site survey, however, is time-consuming and labor-intensive, especially in the case that the RoI is large (e.g., an airport or a big mall). This bottleneck hinders the wide commercial applications of FWIPS (e.g., proximity promotions in a shopping center). To diminish the cost of site survey, we propose a probabilistic model, which combines fingerprinting based positioning (FbP) and RM generation based on stochastic variational Bayesian inference (SVBI). This SVBI based position and RSS estimation has three properties: i) being able to predict the distribution of the estimated position and RSS, ii) treating each observation of RSS at each RP as an example to learn for FbP and RM generation instead of using the whole RM as an example, and iii) requiring only one time training of the SVBI model for both localization and RSS estimation. These benefits make it outperforms the previous proposed approaches.


A fuzzy expert system for earthquake prediction, case study: the Zagros range

arXiv.org Artificial Intelligence

A methodology for the development of a fuzzy expert system (FES) with application to earthquake prediction is presented. The idea is to reproduce the performance of a human expert in earthquake prediction. To do this, at the first step, rules provided by the human expert are used to generate a fuzzy rule base. These rules are then fed into an inference engine to produce a fuzzy inference system (FIS) and to infer the results. In this paper, we have used a Sugeno type fuzzy inference system to build the FES. At the next step, the adaptive network-based fuzzy inference system (ANFIS) is used to refine the FES parameters and improve its performance. The proposed framework is then employed to attain the performance of a human expert used to predict earthquakes in the Zagros area based on the idea of coupled earthquakes. While the prediction results are promising in parts of the testing set, the general performance indicates that prediction methodology based on coupled earthquakes needs more investigation and more complicated reasoning procedure to yield satisfactory predictions.


A Logic for Making Hard Decisions

AAAI Conferences

We tackle the problem of providing engineering decision makers with relevant information extracted from data obtained via a process model based on deliberation and voting. We list examples of potential applications from the area of bug-fix scheduling for software, as well as space-vehicles 'go'-'no-go' decision making. In such application domains, important decisions have to be made hastily and therefore the decision factors have to be informed timely of the main issues discovered by the teams. A logic is proposed for reasoning with comments available in such deliberations. Search based algorithms are proposed which recommend the best justifications for a decision and retain the voting decisions for interested parties to tally. We have developed a Bayesian network for generating data by simulation based on probabilistic models that we can train from collected deliberation databases. The data generated in this way was used for evaluating the proposed search algorithm, showing how it can provide better than random recommendations of arguments to decision makers.


Semantic Web and Ignorance: Dempster-Shafer Description Logics

AAAI Conferences

Information incompleteness, or ignorance, is an issue that we have to consider in Semantic Web applications. Dempster-Shafer theory has been traditionally applied in information incompleteness situations. On the other hand, logic plays a major role in the Semantic Web community. In this paper, we propose a framework that applies Dempster-Shafer theory in a Description Logic Knowledge Base environment. We name our model a Dempster-Shafer DL Knowledge Base.