Most of the existing methods are based on semi-definite programming ( SDP), which is generally time-consuming and degrades the scalability, especially confronting large-scale data. T o overcome this challenge, we propose a stochastic algorithm called SVRG-SBB, which has the following features: i) achieving good scalability via dropping positive semi-definite ( PSD) constraints as serving a fast algorithm, i.e., stochastic variance reduced gradient ( SVRG) method, and ii) adaptive learning via introducing a new, adaptive step size called the stabilized Barzilai-Borwein ( SBB) step size. Theoretically, under some natural assumptions, we show the O ( 1 T) rate of convergence to a stationary point of the proposed algorithm, where T is the number of total iterations. Under the further Polyak- Łojasiewicz assumption, we can show the global linear convergence (i.e., exponentially fast converging to a global optimum) of the proposed algorithm. Numerous simulations and real-world data experiments are conducted to show the effectiveness of the proposed algorithm by comparing with the state-of-the-art methods, notably, much lower computational cost with good prediction performance. Index Terms --Ordinal Embedding, SVRG, Non-Convex Optimization, Barzilai-Borwein (BB) Step Size, .null 1 I NTRODUCTION O RDINAL embedding aims to learn the representation of data as points in a low-dimensional embedded space. Here the "low-dimensional" means the embedding-K. Ma is with the School of Computer Science and T echnology, University of Chinese Academy of Sciences, Beijing 100049, China, and with the Artificial Intelligence Research Center, Peng Cheng Laboratory, Shenzhen 518055, China, and part of this work was performed when he was in the Key Laboratory of Information Security, Institute of Information Engineering, Chinese Academy of Sciences, Beijing 100093, China, and in the School of Cyber Security, University of Chinese Academy of Sciences, Beijing 100049, China. Email: make@ucas.ac.cn - J. Zeng is with the School of Computer Information Engineering, Jiangxi Normal University, Nanchang, Jiangxi 330022, China, and part of this work was performed when he was with the Department of Mathematics, Hong Kong University of Science and T echnology, Clear Water Bay, Kowloon, Hong Kong.

"Time series models are used to forecast future events based on previous events that have been observed (and data collected) at regular time intervals." We will be taking a small forecasting problem and try to solve it till the end learning time series forecasting alongside. Time series forecasting is a technique for the prediction of events through a sequence of time. The technique is used across many fields of study, from geology to behavior to economics. The techniques predict future events by analyzing the trends of the past, on the assumption that future trends will hold similar to historical trends.

In September 2018, I published a blog about my forthcoming book on The Mathematical Foundations of Data Science. The central question we address is: How can we bridge the gap between mathematics needed for Artificial Intelligence (Deep Learning and Machine learning) with that taught in high schools (up to ages 17/18)? In this post, we present a chapter from this book called "A Taxonomy of Machine Learning Models." The book is now available for an early bird discount released as chapters. If you are interested in getting early discounted copies, please contact ajit.jaokar at feynlabs.ai.

We present a learning-based force-torque dynamics to achieve model-based control for contact-rich peg-in-hole task using force-only inputs. Learning the force-torque dynamics is challenging because of the ambiguity of the low-dimensional 6-d force signal and the requirement of excessive training data. To tackle these problems, we propose a multi-pose force-torque state representation, based on which a dynamics model is learned with the data generated in a sample-efficient offline fashion. In addition, by training the dynamics model with peg-and-holes of various shapes, scales, and elasticities, the model could quickly transfer to new peg-and-holes after a small number of trials. Extensive experiments show that our dynamics model could adapt to unseen peg-and-holes with 70% fewer samples required compared to learning from scratch. Along with the learned dynamics, model predictive control and model-based reinforcement learning policies achieve over 80% insertion success rate. Our video is available at https://youtu.be/ZAqldpVZgm4.

Automatic dialog act recognition is a task that has been widely explored over the years. In recent works, most approaches to the task explored different deep neural network architectures to combine the representations of the words in a segment and generate a segment representation that provides cues for intention. In this study, we explore means to generate more informative segment representations, not only by exploring different network architectures, but also by considering different token representations, not only at the word level, but also at the character and functional levels. At the word level, in addition to the commonly used uncontextualized embeddings, we explore the use of contextualized representations, which are able to provide information concerning word sense and segment structure. Character-level tokenization is important to capture intention-related morphological aspects that cannot be captured at the word level. Finally, the functional level provides an abstraction from words, which shifts the focus to the structure of the segment. Additionally, we explore approaches to enrich the segment representation with context information from the history of the dialog, both in terms of the classifications of the surrounding segments and the turn-taking history. This kind of information has already been proved important for the disambiguation of dialog acts in previous studies. Nevertheless, we are able to capture additional information by considering a summary of the dialog history and a wider turn-taking context. By combining the best approaches at each step, we achieve performance results that surpass the previous state-of-the-art on generic dialog act recognition on both the Switchboard Dialog Act Corpus (SwDA) and the ICSI Meeting Recorder Dialog Act Corpus (MRDA), which are two of the most widely explored corpora for the task. Furthermore, by considering both past and future context, similarly to what happens in an annotation scenario, our approach achieves a performance similar to that of a human annotator on SwDA and surpasses it on MRDA.

This work aims to examine one of the cornerstone problems of Musical Instrument Recognition, in particular instrument classification. IRMAS (Instrument recognition in Musical Audio Signals) data set is chosen. The data includes music obtained from various decades in the last century, thus having a wide variety in audio quality. We have presented a very concise summary of past work in this domain. Having implemented various supervised learning algorithms for this classification task, SVM classifier has outperformed the other state-of-the-art models with an accuracy of 79%. The classifier had a major challenge distinguishing between flute and organ. We also implemented Unsupervised techniques out of which Hierarchical Clustering has performed well. We have included most of the code (jupyter notebook) for easy reproducibility.

Transfer learning makes use of data or knowledge in one task to help solve a different, yet related, task. Many ex isting TL approaches are based on a joint probability distribution metric, which is a weighted sum of the marginal distribution and the c ondi-tional distribution; however, they optimize the two distri butions independently, and ignore their intrinsic dependency. This p aper proposes a novel and frustratingly easy Joint Probability Dist ribution Adaptation (JPDA) approach, to replace the frequently-use d joint maximum mean discrepancy metric in transfer learning. Duri ng the distribution adaptation, JPDA improves the transferabili ty between the source and the target domains by minimizing the joint pro b-ability discrepancy of the corresponding class, and also in creases the discriminability between different classes by maximiz ing their joint probability discrepancy. Experiments on six image cl assifica-tion datasets demonstrated that JPDA outperforms several s tate-of- the-art metric-based transfer learning approaches.

--In this paper we introduce Anomaly Contribution Explainer or ACE, a tool to explain security anomaly detection models in terms of the model features through a regression framework, and its variant, ACE-KL, which highlights the important anomaly contributors. ACE and ACE-KL provide insights in diagnosing which attributes significantly contribute to an anomaly by building a specialized linear model to locally approximate the anomaly score that a black-box model generates. We conducted experiments with these anomaly detection models to detect security anomalies on both synthetic data and real data. In particular, we evaluate performance on three public data sets: CERT insider threat, netflow logs, and Android malware. The experimental results are encouraging: our methods consistently identify the correct contributing feature in the synthetic data where ground truth is available; similarly, for real data sets, our methods point a security analyst in the direction of the underlying causes of an anomaly, including in one case leading to the discovery of previously overlooked network scanning activity. We have made our source code publicly available. Cyber-security is a key concern for both private and public organizations, given the high cost of security compromises and attacks; malicious cyber-activity cost the U.S. economy between $57 billion and $109 billion in 2016 [1]. As a result, spending on security research and development, and security products and services to detect and combat cyber-attacks has been increasing [2]. Organizations produce large amounts of network, host and application data that can be used to gain insights into cyber-security threats, misconfigurations, and network operations. While security domain experts can manually sift through some amount of data to spot attacks and understand them, it is virtually impossible to do so at scale, considering that even a medium sized enterprise can produce terabytes of data in a few hours.

A new semi-supervised ensemble algorithm called XGBOD (Extreme Gradient Boosting Outlier Detection) is proposed, described and demonstrated for the enhanced detection of outliers from normal observations in various practical datasets. The proposed framework combines the strengths of both supervised and unsupervised machine learning methods by creating a hybrid approach that exploits each of their individual performance capabilities in outlier detection. XGBOD uses multiple unsupervised outlier mining algorithms to extract useful representations from the underlying data that augment the predictive capabilities of an embedded supervised classifier on an improved feature space. The novel approach is shown to provide superior performance in comparison to competing individual detectors, the full ensemble and two existing representation learning based algorithms across seven outlier datasets.

Due to technological advances, large and high-dimensional data have become the rule rather than the exception. Methods that allow for feature selection with s uch data are thus highly sought after, in particular, since standard methods, such as cro ss-validated lasso and group-lasso, can be challenging both computationally and mathematically. In this paper, we propose a novel approach to feature selection and group feature selection in linear regression. It consists of simple optimization steps and tests, which makes it com putationally more efficient than standard approaches and suitable even for very larg e data sets. Moreover, it satisfies sharp guarantees for estimation and feature selection in terms of oracle inequalities. We thus expect that our contribution can help to leverage the incre asing volume of data in Biology, Public Health, Astronomy, Economics, and other fields.