Statistical Learning
Learning Binary Trees via Sparse Relaxation
Zantedeschi, Valentina, Kusner, Matt J., Niculae, Vlad
One of the most classical problems in machine learning is how to learn binary trees that split data into useful partitions. From classification/regression via decision trees to hierarchical clustering, binary trees are useful because they (a) are often easy to visualize; (b) make computationally-efficient predictions; and (c) allow for flexible partitioning. Because of this there has been extensive research on how to learn such trees that generally fall into one of three categories: 1. greedy node-by-node optimization; 2. probabilistic relaxations for differentiability; 3. mixed-integer programs (MIP). Each of these have downsides: greedy can myopically choose poor splits, probabilistic relaxations do not have principled ways to prune trees, MIP methods can be slow on large problems and may not generalize. In this work we derive a novel sparse relaxation for binary tree learning. By deriving a new MIP and sparsely relaxing it, our approach is able to learn tree splits and tree pruning using argmin differentiation. We demonstrate how our approach is easily visualizable and is competitive with current tree-based approaches in classification/regression and hierarchical clustering. Source code is available at http://github.com/vzantedeschi/LatentTrees .
HENIN: Learning Heterogeneous Neural Interaction Networks for Explainable Cyberbullying Detection on Social Media
In the computational detection of cyberbullying, existing work largely focused on building generic classifiers that rely exclusively on text analysis of social media sessions. Despite their empirical success, we argue that a critical missing piece is the model explainability, i.e., why a particular piece of media session is detected as cyberbullying. In this paper, therefore, we propose a novel deep model, HEterogeneous Neural Interaction Networks (HENIN), for explainable cyberbullying detection. HENIN contains the following components: a comment encoder, a post-comment co-attention sub-network, and session-session and post-post interaction extractors. Extensive experiments conducted on real datasets exhibit not only the promising performance of HENIN, but also highlight evidential comments so that one can understand why a media session is identified as cyberbullying.
Is Standard Deviation the New Standard? Revisiting the Critic in Deep Policy Gradients
Flet-Berliac, Yannis, Ouhamma, Reda, Maillard, Odalric-Ambrym, Preux, Philippe
Policy gradient algorithms have proven to be successful in diverse decision making and control tasks. However, these methods suffer from high sample complexity and instability issues. In this paper, we address these challenges by providing a different approach for training the critic in the actor-critic framework. Our work builds on recent studies indicating that traditional actor-critic algorithms do not succeed in fitting the true value function, calling for the need to identify a better objective for the critic. In our method, the critic uses a new state-value (resp. state-action-value) function approximation that learns the relative value of the states (resp. state-action pairs) rather than their absolute value as in conventional actor-critic. We prove the theoretical consistency of the new gradient estimator and observe dramatic empirical improvement across a variety of continuous control tasks and algorithms. Furthermore, we validate our method in tasks with sparse rewards, where we provide experimental evidence and theoretical insights.
A Survey of Knowledge-Enhanced Text Generation
Yu, Wenhao, Zhu, Chenguang, Li, Zaitang, Hu, Zhiting, Wang, Qingyun, Ji, Heng, Jiang, Meng
The goal of text generation is to make machines express in human language. It is one of the most important yet challenging tasks in natural language processing (NLP). Since 2014, various neural encoder-decoder models pioneered by Seq2Seq have been proposed to achieve the goal by learning to map input text to output text. However, the input text alone often provides limited knowledge to generate the desired output, so the performance of text generation is still far from satisfaction in many real-world scenarios. To address this issue, researchers have considered incorporating various forms of knowledge beyond the input text into the generation models. This research direction is known as knowledge-enhanced text generation. In this survey, we present a comprehensive review of the research on knowledge enhanced text generation over the past five years. The main content includes two parts: (i) general methods and architectures for integrating knowledge into text generation; (ii) specific techniques and applications according to different forms of knowledge data. This survey can have broad audiences, researchers and practitioners, in academia and industry.
Machine Learning Applied to Registry Data
Craniosynostosis is the premature fusion of 1 cranial sutures and often requires surgical intervention. Surgery may involve extensive osteotomies, which can lead to substantial blood loss. Currently, there are no consensus recommendations for guiding blood conservation or transfusion in this patient population. The aim of this study is to develop a machine-learning model to predict blood product transfusion requirements for individual pediatric patients undergoing craniofacial surgery. Using data from 2143 patients in the Pediatric Craniofacial Surgery Perioperative Registry, we assessed 6 machine-learning classification and regression models based on random forest, adaptive boosting (AdaBoost), neural network, gradient boosting machine (GBM), support vector machine, and elastic net methods with inputs from 22 demographic and preoperative features.
The IBM Data Scientist Interview
IBM is a multinational technology company founded in 1911 and operates in over 170 countries worldwide. Today, IBM offers a wide spectrum of products and services that includes software solutions, hardware architecture (server and storage architecture), business and technology services, and global financing solutions. As a data driven-company, IBM understands the importance of data and data analytics at every layer of organization to drive better business decisions. Also, a leading provider of Analytics and Cloud-based solutions, IBM offers a full stack of cloud-based products and services spanning across data analytics, storage, AI, IoT, and blockchain. Check out this article about the Microsoft Data Scientist interview!
Response to Comment on "Females engaging in adaptive hybridization prefer high-quality heterospecifics as mates"
Braun et al. contend that we did not account for survival, but we did. Differential survival does not alter our conclusions, which were also robust to removing anomalous families. They ignore the study system's natural history justifying our fitness measures, while failing to account for our behavioral data. We stand by our conclusion that females adaptively choose among heterospecific males. Hybridization is adaptive if its fitness benefits outweigh its costs (1).
Classical Machine Learning
Machine Learning (ML) initially started in the '50s and '60s as pattern recognition. They got better by seeing more data. Most of these algorithms were based on statistics and probabilistic reasoning, by measuring the distance between data points, directions of vectors, intensities of values, etc. These [old] algorithms are widespread. You can see them everywhere -- read this article next, Google reporting your random sign-in -- they're simple to use, easy to understand, and don't require a large amount of computational power.
Why Data Scientists Should Learn Algorithms and Data Structures?
The first useful concept you will encounter is algorithmic complexity and Big-Oh notation. It is a method that allows understanding how well your code scales with the data. This concept is important to data scientists due to the need to process an ever-increasing amount of information produced daily. By getting rid of less important details, you will be able to reason about the performance of an algorithm regardless if it is written in Python or C and executed on a laptop or NASA's supercomputer. In a sense, it defines a basic vocabulary for the design and analysis of algorithms while suppressing architecture and language-dependent details -- these are considered a constant factor, not relevant to the big picture.
The 13 Best Machine Learning Courses and Online Training for 2020
The editors at Solutions Review have compiled this list of the best machine learning courses and online training to consider for 2020. Description: This course provides a broad introduction to machine learning, datamining, and statistical pattern recognition. Topics include: (i) Supervised learning (parametric/non-parametric algorithms, support vector machines, kernels, neural networks). Description: In this non-technical course, you'll learn everything you've been too afraid to ask about machine learning. Hands-on exercises will help you get past the jargon and learn how this exciting technology powers everything from self-driving cars to your personal Amazon shopping suggestions.