Several studies have shown that combining machine learning models in an appropriate way will introduce improvements in the individual predictions made by the base models. The key to make well-performing ensemble model is in the diversity of the base models. Of the most common solutions for introducing diversity into the decision trees are bagging and random forest. Bagging enhances the diversity by sampling with replacement and generating many training data sets, while random forest adds selecting a random number of features as well. This has made the random forest a winning candidate for many machine learning applications. However, assuming equal weights for all base decision trees does not seem reasonable as the randomization of sampling and input feature selection may lead to different levels of decision-making abilities across base decision trees. Therefore, we propose several algorithms that intend to modify the weighting strategy of regular random forest and consequently make better predictions. The designed weighting frameworks include optimal weighted random forest based on ac-curacy, optimal weighted random forest based on the area under the curve (AUC), performance-based weighted random forest, and several stacking-based weighted random forest models. The numerical results show that the proposed models are able to introduce significant improvements compared to regular random forest.

Time-varying parameters (TVPs) models are frequently used in economics to model structural change. I show that they are in fact ridge regressions. Instantly, this makes computations, tuning, and implementation much easier than in the state-space paradigm. Among other things, solving the equivalent dual ridge problem is computationally very fast even in high dimensions, and the crucial "amount of time variation" is tuned by cross-validation. Evolving volatility is dealt with using a two-step ridge regression. I consider extensions that incorporate sparsity (the algorithm selects which parameters vary and which do not) and reduced-rank restrictions (variation is tied to a factor model). To demonstrate the usefulness of the approach, I use it to study the evolution of monetary policy in Canada. The application requires the estimation of about 4600 TVPs, a task well within the reach of the new method.

We recently launched Elastic Security, combining the threat hunting and analytics tools from Elastic SIEM with the prevention and response features of Elastic Endpoint Security. This combined solution focuses on detecting and flexibly responding to security threats, with machine learning providing core capabilities for real-time protections, detections, and interactive hunting. But why are machine learning tools so important in information security? How is machine learning being applied? In this first of a two-part blog series, we'll motivate the "why" and explore the "how," highlighting malware prevention via supervised machine learning in Elastic Endpoint Security.

Working with object detection in deep learning can sometimes be difficult but it is extremely fun. And it feels even more rewarding when we are able to solve the detection problem on our custom dataset. But when working with a custom dataset, how do we know that our object detection algorithm has reached its optimal performance? How do we know that going through a few more epochs won't lead to overfitting of the deep learning algorithm? In fact, these questions are really important when working to competitive datasets like PASCAL VOC or MS COCO dataset as well. These competition datasets have pretty stringent object detection evaluation metrics.

It is really challenging to improve existing visualization methods or to transport methods from other research fields. You have to think about the dimensions in your plot and the ways to add more of them. A good example is the path from a boxplot to a violinplot to a swarmplot. It is a continuous process of adding dimensions and thus information. Categories can be added with different marker shapes, color maps like in a heat map can serve as another dimension and the size of a marker can give insight to further parameters.

Protein contacts provide key information for the understanding of protein structure and function, and therefore contact prediction from sequences is an important problem. Recent research shows that some correctly predicted long-range contacts could help topology-level structure modeling. Thus, contact prediction and contact-assisted protein folding also proves the importance of this problem. In this thesis, I will briefly introduce the extant related work, then show how to establish the contact prediction through unsupervised graphical models with topology constraints. Further, I will explain how to use the supervised deep learning methods to further boost the accuracy of contact prediction. Finally, I will propose a scoring system called diversity score to measure the novelty of contact predictions, as well as an algorithm that predicts contacts with respect to the new scoring system.

As the name says, these approaches rely on the availability of data to extract knowledge and train algorithms. This is opposed to, e.g., modeling approaches in which physiological, physics-based, mathematical, and other equations form the basis of algorithms, or, rule-based systems in which reasoning processes are obtained by translating domain-experts' knowledge into computer-based rules. Focusing on data-driven systems, the data plays a role in several components during the development and actual usage phases. First, we need data to extract knowledge from, i.e., to develop and train algorithms so that they learn-by-example the properties of the problem at hand and get better at solving the problem by repeatedly providing example data. Second, we need to monitor during the development phase how promising the algorithms are and make choices, e.g., concerning optimisation of parameters or choosing different MLparadigms. Methods that don't perform well at all can be discarded, and ones that seem promising can be further optimised. To assess how promising a specific method is, we need to examine how it performs on data that was not used during training. Finally, to objectively assess how well the final'best' system performs, we need to apply completely new data to it that has not been used at all thusfar during the research and development process. Thus, there are at least three stakeholders that have the interest to get as large part of the data pie as possible.

Millions of mosques around the world are suffering some problems such as ventilation and difficulty getting rid of bacteria, especially in rush hours where congestion in mosques leads to air pollution and spread of bacteria, in addition to unpleasant odors and to a state of discomfort during the pray times, where in most mosques there are no enough windows to ventilate the mosque well. This paper aims to solve these problems by building a model of smart mosques domes using weather features and outside temperatures. Machine learning algorithms such as k Nearest Neighbors and Decision Tree were applied to predict the state of the domes open or close. The experiments of this paper were applied on Prophet mosque in Saudi Arabia, which basically contains twenty seven manually moving domes. Both machine learning algorithms were tested and evaluated using different evaluation methods. After comparing the results for both algorithms, DT algorithm was achieved higher accuracy 98% comparing with 95% accuracy for kNN algorithm. Finally, the results of this study were promising and will be helpful for all mosques to use our proposed model for controlling domes automatically.

Various studies have shown that students tend to get higher marks when assessed through coursework based assessment methods which include either modules that are fully assessed through coursework or a mixture of coursework and examinations than assessed by examination alone. There are a large number of educational data mining studies that preprocess data through conventional data mining processes including data preparation process, but they are using transcript data as they stand without looking at examination and coursework results weighting which could affect prediction accuracy. This paper proposes a different data preparation process through investigating more than 230000 student records in order to prepare students marks based on the assessment methods of enrolled modules. The data have been processed through different stages in order to extract a categorical factor through which students module marks are refined during the data preparation process. The results of this work show that students final marks should not be isolated from the nature of the enrolled modules assessment methods. They must rather be investigated thoroughly and considered during EDMs data preprocessing phases. More generally, it is concluded that educational data should not be prepared in the same way as other data types due to differences as data sources, applications, and types of errors in them. Therefore, an attribute, coursework assessment ratio, is proposed to be used in order to take the different modules assessment methods into account while preparing student transcript data. The effect of CAR on prediction process using the random forest classification technique has been investigated. It is shown that considering CAR as an attribute increases the accuracy of predicting students second year averages based on their first year results.

Dense embedding models are commonly deployed in commercial search engines, wherein all the document vectors are pre-computed, and near-neighbor search (NNS) is performed with the query vector to find relevant documents. However, the bottleneck of indexing a large number of dense vectors and performing an NNS hurts the query time and accuracy of these models. In this paper, we argue that high-dimensional and ultra-sparse embedding is a significantly superior alternative to dense low-dimensional embedding for both query efficiency and accuracy. Extreme sparsity eliminates the need for NNS by replacing them with simple lookups, while its high dimensionality ensures that the embeddings are informative even when sparse. However, learning extremely high dimensional embeddings leads to blow up in the model size. To make the training feasible, we propose a partitioning algorithm that learns such high dimensional embeddings across multiple GPUs without any communication. This is facilitated by our novel asymmetric mixture of Sparse, Orthogonal, Learned and Random (SOLAR) Embeddings. The label vectors are random, sparse, and near-orthogonal by design, while the query vectors are learned and sparse. We theoretically prove that our way of one-sided learning is equivalent to learning both query and label embeddings. With these unique properties, we can successfully train 500K dimensional SOLAR embeddings for the tasks of searching through 1.6M books and multi-label classification on the three largest public datasets. We achieve superior precision and recall compared to the respective state-of-the-art baselines for each of the tasks with up to 10 times faster speed.