Bayesian Learning
Bayesian Estimation of Nelson-Siegel model using rjags R package
To leave a comment for the author, please follow the link and comment on their blog: K & L Fintech Modeling. R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job. Want to share your content on R-bloggers? click here if you have a blog, or here if you don't. To leave a comment for the author, please follow the link and comment on their blog: K & L Fintech Modeling.
S-DABT: Schedule and Dependency-Aware Bug Triage in Open-Source Bug Tracking Systems
Jahanshahi, Hadi, Cevik, Mucahit
Fixing bugs in a timely manner lowers various potential costs in software maintenance. However, manual bug fixing scheduling can be time-consuming, cumbersome, and error-prone. In this paper, we propose the Schedule and Dependency-aware Bug Triage (S-DABT), a bug triaging method that utilizes integer programming and machine learning techniques to assign bugs to suitable developers. Unlike prior works that largely focus on a single component of the bug reports, our approach takes into account the textual data, bug fixing costs, and bug dependencies. We further incorporate the schedule of developers in our formulation to have a more comprehensive model for this multifaceted problem. As a result, this complete formulation considers developers' schedules and the blocking effects of the bugs while covering the most significant aspects of the previously proposed methods. Our numerical study on four open-source software systems, namely, EclipseJDT, LibreOffice, GCC, and Mozilla, shows that taking into account the schedules of the developers decreases the average bug fixing times. We find that S-DABT leads to a high level of developer utilization through a fair distribution of the tasks among the developers and efficient use of the free spots in their schedules. Via the simulation of the issue tracking system, we also show how incorporating the schedule in the model formulation reduces the bug fixing time, improves the assignment accuracy, and utilizes the capability of each developer without much comprising in the model run times. We find that S-DABT decreases the complexity of the bug dependency graph by prioritizing blocking bugs and effectively reduces the infeasible assignment ratio due to bug dependencies. Consequently, we recommend considering developers' schedules while automating bug triage.
Automated Learning of Interpretable Models with Quantified Uncertainty
Bomarito, G. F., Leser, P. E., Strauss, N. C. M, Garbrecht, K. M., Hochhalter, J. D.
Machine learning (ML) has become ubiquitous in scientific disciplines. In some applications, accurate data-driven predictions are all that is required; however, in many others, interpretability and explainability of the model is equally important. Interpretability and explainability can provide justification for decisions, promote scientific discovery and ultimately lead to better control/improvement of models [1, 2]. In a complementary fashion, ML models can provide further insight by conveying their level of uncertainty in predictions [3]. Especially in cases of low risk tolerance this type of insight is crucial for building trust in ML models [4]. Rather than focus on black-box ML methods (e.g., neural networks or Gaussian process regression) combined with post hoc explainability tools, the current work focuses on inherently interpretable methods. Interpretable ML methods can be competitive with black-box ML in terms of accuracy and do not require a separate explainability toolkit [4, 5]. Symbolic regression is one such inherently interpretable form of ML wherein an analytic equation is produced that best models input data.
Mathematics for Deep Learning (Part 7)
In the road so far, we have talked about MLP, CNN, and RNN architectures. These are discriminative models, that is models that can make predictions. Discriminative models essentially learn to estimate a conditional probability distribution p( x); that is, given a value, they try to predict the outcome based on what they learned about the probability distribution of x. Generative models are architectures of neural networks that learn the probability distribution of the data and learn how to generate data that seems to come from that probability distribution. Creating synthetic data is one use of generative models, but is not the only one.
On unsupervised projections and second order signals
Lartigue, Thomas, Mukherjee, Sach
Linear projections are widely used in the analysis of high-dimensional data. In unsupervised settings where the data harbour latent classes/clusters, the question of whether class discriminatory signals are retained under projection is crucial. In the case of mean differences between classes, this question has been well studied. However, in many contemporary applications, notably in biomedicine, group differences at the level of covariance or graphical model structure are important. Motivated by such applications, in this paper we ask whether linear projections can preserve differences in second order structure between latent groups. We focus on unsupervised projections, which can be computed without knowledge of class labels. We discuss a simple theoretical framework to study the behaviour of such projections which we use to inform an analysis via quasi-exhaustive enumeration. This allows us to consider the performance, over more than a hundred thousand sets of data-generating population parameters, of two popular projections, namely random projections (RP) and Principal Component Analysis (PCA). Across this broad range of regimes, PCA turns out to be more effective at retaining second order signals than RP and is often even competitive with supervised projection. We complement these results with fully empirical experiments showing 0-1 loss using simulated and real data. We study also the effect of projection dimension, drawing attention to a bias-variance trade-off in this respect. Our results show that PCA can indeed be a suitable first-step for unsupervised analysis, including in cases where differential covariance or graphical model structure are of interest.
Multinomial Naัve Bayes' For Documents Classification and Natural Language Processing (NLP)
It's formulated as several methods, widely used as an alternative to the distance-based K-Means clustering and decision tree forests, and deals with probability as the "likelihood" that data belongs to a specific class. The Gaussian and Multinomial models of the naรฏve Bayes exist. The multinomial model provides an ability to classify data, that cannot be represented numerically. Its main advantage is the significantly reduced complexity. It provides an ability to perform the classification, using small training sets, not requiring to be continuously re-trained.
Information-theoretic Online Memory Selection for Continual Learning
Sun, Shengyang, Calandriello, Daniele, Hu, Huiyi, Li, Ang, Titsias, Michalis
A challenging problem in task-free continual learning is the online selection of a representative replay memory from data streams. In this work, we investigate the online memory selection problem from an information-theoretic perspective. To gather the most information, we propose the surprise and the learnability criteria to pick informative points and to avoid outliers. We present a Bayesian model to compute the criteria efficiently by exploiting rank-one matrix structures. We demonstrate that these criteria encourage selecting informative points in a greedy algorithm for online memory selection. Furthermore, by identifying the importance of the timing to update the memory, we introduce a stochastic informationtheoretic reservoir sampler (InfoRS), which conducts sampling among selective points with high information. Compared to reservoir sampling, InfoRS demonstrates improved robustness against data imbalance. Continual learning (Robins, 1995; Goodfellow et al., 2013; Kirkpatrick et al., 2017) aims at training models through a non-stationary data stream without catastrophic forgetting of past experiences. Specifically, replay-based methods (Lopez-Paz & Ranzato, 2017; Rebuffi et al., 2017; Rolnick et al., 2019) tackle the continual learning problem by keeping a replay memory for rehearsals over the past data. Given the limited memory budget, selecting a representative memory becomes critical. The majority of existing approaches focus on task-based continual learning and update the memory based on the given task boundaries. Since the requirement for task boundaries is usually not realistic, general continual learning (GCL) (Aljundi et al., 2019a; Delange et al., 2021; Buzzega et al., 2020) has received increasing attention, which assumes that the agent observes the streaming data in an online fashion without knowing task boundaries. GCL makes the online memory selection more challenging since one needs to update the memory in each iteration based only on instant observations. So, successful memory management for GCL needs to be both efficient and effective.
RMFGP: Rotated Multi-fidelity Gaussian process with Dimension Reduction for High-dimensional Uncertainty Quantification
Zhang, Jiahao, Zhang, Shiqi, Lin, Guang
Multi-fidelity modelling arises in many situations in computational science and engineering world. It enables accurate inference even when only a small set of accurate data is available. Those data often come from a high-fidelity model, which is computationally expensive. By combining the realizations of the high-fidelity model with one or more low-fidelity models, the multi-fidelity method can make accurate predictions of quantities of interest. This paper proposes a new dimension reduction framework based on rotated multi-fidelity Gaussian process regression and a Bayesian active learning scheme when the available precise observations are insufficient. By drawing samples from the trained rotated multi-fidelity model, the so-called supervised dimension reduction problems can be solved following the idea of the sliced average variance estimation (SAVE) method combined with a Gaussian process regression dimension reduction technique. This general framework we develop can effectively solve high-dimensional problems while the data are insufficient for applying traditional dimension reduction methods. Moreover, a more accurate surrogate Gaussian process model of the original problem can be obtained based on our trained model. The effectiveness of the proposed rotated multi-fidelity Gaussian process(RMFGP) model is demonstrated in four numerical examples. The results show that our method has better performance in all cases and uncertainty propagation analysis is performed for last two cases involving stochastic partial differential equations.
How is Maximum Likelihood Estimation used in machine learning?
Maximum Likelihood Estimation (MLE) is a probabilistic based approach to determine values for the parameters of the model. Parameters could be defined as blueprints for the model because based on that the algorithm works. MLE is a widely used technique in machine learning, time series, panel data and discrete data. The motive of MLE is to maximize the likelihood of values for the parameter to get the desired outcomes. Following are the topics to be covered.
Top Machine Learning Algorithms Used By AI Professionals: Explained
Machine Learning and Artificial Intelligence have been deemed the "hot topics" for every trending article in 2021. It's similar to how the internet revolutionized everyone's lives. Artificial Intelligence (A.I.) and Machine Learning will transform our lives in ways labelled impossible years ago. In 1959, Arthur Samuel coined the term Machine Learning. He was a pioneer in Artificial Intelligence, computer gaming and Machine Learning.