The connection between machine learning and statistical physics has long been appreciated [1-9], but deeper relationships continue to be uncovered. For example, techniques used to pre-train neural networks [8] have more recently been interpreted in terms of the renor-malization group [10]. In the other direction there has been a sharp increase in applications of machine learning to chemistry, material science, and condensed matter physics [11-19], which are sources of highly-structured data and could be a good testing ground for machine learning techniques. A recent trend in both physics and machine learning is an appreciation for the power of tensor methods. In machine learning, tensor decompositions can be used to solve non-convex optimization tasks [20, 21] and make progress on many other important problems [22-24], while in physics, great strides have been made in manipulating large vectors arising in quantum mechanics by decomposing them as tensor networks [25-27]. The most successful types of tensor networks avoid the curse of dimensionality by incorporating only low-order tensors, yet accurately reproduce very high-order tensors through a particular geometry of tensor contractions [27]. Another context where very large vectors arise is in nonlinear kernel learning, where input vectors x are mapped into a higher dimensional space via a feature map Φ( x) before being classified by a decision function f ( x) W · Φ( x).

Text mining (deriving information from text) is a wide field which has gained popularity with the huge text data being generated. Automation of a number of applications like sentiment analysis, document classification, topic classification, text summarization, machine translation, etc has been done using machine learning models. Spam filtering is a beginner's example of document classification task which involves classifying an email as spam or non-spam (a.k.a. Spam box in your Gmail account is the best example of this. So lets get started in building a spam filter on a publicly available mail corpus.

Who will remain a loyal customer and who won't? Which messages are most effective with which segments? How can customer value be maximized? This book supplies powerful tools for extracting the answers to these and other crucial business questions from the corporate databases where they lie buried. In the years since the first edition of this book, data mining has grown to become an indispensable tool of modern business.

Abstract-- The concept of ensemble learning offers a promising avenue in learning from data streams under complex environments because it addresses the bias and variance dilemma better than its single-model counterpart and features a reconfigurable structure, which is well-suited to the given context. While various extensions of ensemble learning for mining nonstationary data streams can be found in the literature, most of them are crafted under a static base-classifier and revisits preceding samples in the sliding window for a retraining step. This feature causes computationally prohibitive complexity and is not flexible enough to cope with rapidly changing environments. Their complexities are often demanding because it involves a large collection of offline classifiers due to the absence of structural complexities reduction mechanisms and lack of an online feature selection mechanism. A novel evolving ensemble classifier, namely Parsimonious Ensemble (pENsemble), is proposed in this paper. A dynamic online feature selection scenario is integrated into the pENsemble. This method allows for dynamic selection and deselection of input features on the fly. The efficacy of the pENsemble has been numerically demonstrated through rigorous numerical studies with dynamic and evolving data streams where it delivers the most encouraging performance in attaining a tradeoff between accuracy and complexity. I. INTRODUCTION The data-intensive era where data are collected continuously in a fast rate under dynamic and evolving environments opens a new research direction to process data streams efficiently [1], [2]. Unlike a classical paradigm in machine learning where a dataset is utilised to construct hypothesis and is executed over multiple passes, data streams requires a strictly online learning framework with a low memory requirement and even if possible with no memory at all - one-pass learning mode. Another challenging trait of data streams lies in the non-stationary characteristics [3] where the data does not follow static and predictable distributions and contains a variety of concept drifts [4], [5]. These facts make a retraining phase when incorporating a new sample to an old dataset impossible to be performed because it leads to the socalled catastrophic forgetting [6] of previously valid knowledge and is not scalable when dealing with massive data streams. Evolving Intelligent System (EIS) provides a unique solution for data stream mining because a strictly one-pass learning procedure involved here has delivered great success to cope with time-critical applications where data streams are generated at a very fast sampling rate [7]. Furthermore, EIS adopts an open structure where its components can be automatically generated, pruned, merged and recalled on the fly [8], [9] and can be well-suited to a given problem.

If there are n number of instances in data, probability of'success' is 1/n and for the failure, its (n-1)/n. In the specific case of a bootstrap sample, the sample size b equals the number of instances n. Thus the probability of the instance being selected atleast once is 1-1/e 0.632 Grid search is computationally expensive as it checks for all the possible combinations of the parameters specified and evaluates on the same. Lets say if two parameters are A and B, and the possible ranges specified are 0-2 and 0-3 respectively; The possible combinations in the parameter space in case of grid search would be (0,0) (0,1) (0,2) (0,3) ...........(2,2) (2,3). Although grid search can be made to run in parallel, still the technique is not computationally very efficient .

Machine learning automatically recognizes complex, previously unknown, novel, and useful patterns and information in all types of data. Data driven algorithms are the wave of the future and their results improve as the amount of data increases. Machine learning algorithms are used in search engines, image analysis, multimedia database retrieval, bioinformatics, industrial automation, speech recognition, and many other fields. This survey course covers the concepts and principles of a large variety of data mining methods, equips you with a working knowledge of these techniques and prepares you to apply them to real problems. The statistical programming language R is used to implement machine learning algorithms.

Whether you are a newcomer to machine learning, a newbie to specific algorithms or concepts, or a seasoned ML vet looking for a once-over of an algorithm you haven't seen or used in a while, these short and to-the-point tutorials may provide the assistance you are looking for. Each of these posts concisely covers a single, specific machine learning concept. Support Vector Machines remain a popular and time-tested classification algorithm. This post provides a high-level concise technical overview of their functionality. A wide array of clustering techniques are in use today.

In this work, we present a methodology that enables an agent to make efficient use of its exploratory actions by autonomously identifying possible objectives in its environment and learning them in parallel. The identification of objectives is achieved using an online and unsupervised adaptive clustering algorithm. The identified objectives are learned (at least partially) in parallel using Q-learning. Using a simulated agent and environment, it is shown that the converged or partially converged value function weights resulting from off-policy learning can be used to accumulate knowledge about multiple objectives without any additional exploration. We claim that the proposed approach could be useful in scenarios where the objectives are initially unknown or in real world scenarios where exploration is typically a time and energy intensive process. The implications and possible extensions of this work are also briefly discussed.

Randomized dimensionality reduction techniques, such as random projection (RP) [7, 15] and Ho's random subspace method (RS) [12] are popular approaches for data compression, with many empirical studies showing the utility of both for machine learning and data mining tasks in practice [26, 11, 21, 19, 18, 27]. For RP a key theoretical motivation behind their use is the Johnson-Lindenstrauss lemma (JLL), the usual constructive proof of which also implies an algorithm with high-probability geometry preservation guarantees for projected data. However RP is costly to apply to large or high-dimensional datasets since it requires a matrix-matrix multiplication to implement the projection, and furthermore the projected features may be hard to interpret. On the other hand RS is a particularly appealing approach for dimensionality reduction because it involves simply selecting a subset of data feature indices randomly without replacement, and so does not require a matrix-matrix multiplication to implement the projection and it retains (a subset of) the original features. RS is therefore computationally far more efficient in practice, and more interpretable than RP, but there is little theory to explain its effectiveness. Focusing on this latter problem, here we prove data-dependent norm-preservation guarantees for data projected onto a random subset of the data features.

Recent several years have witnessed the surge of asynchronous (async-) parallel computing methods due to the extremely big data involved in many modern applications and also the advancement of multi-core machines and computer clusters. In optimization, most works about async-parallel methods are on unconstrained problems or those with block separable constraints. In this paper, we propose an async-parallel method based on block coordinate update (BCU) for solving convex problems with nonseparable linear constraint. Running on a single node, the method becomes a novel randomized primal-dual BCU with adaptive stepsize for multi-block affinely constrained problems. For these problems, Gauss-Seidel cyclic primal-dual BCU needs strong convexity to have convergence. On the contrary, merely assuming convexity, we show that the objective value sequence generated by the proposed algorithm converges in probability to the optimal value and also the constraint residual to zero. In addition, we establish an ergodic $O(1/k)$ convergence result, where $k$ is the number of iterations. Numerical experiments are performed to demonstrate the efficiency of the proposed method and significantly better speed-up performance than its sync-parallel counterpart.