Stock return predictability is an important research theme as it reflects our economic and social organization, and significant efforts are made to explain the dynamism therein. Statistics of strong explanative power, called "factor" have been proposed to summarize the essence of predictive stock returns. Although machine learning methods are increasingly popular in stock return prediction, an inference of the stock returns is highly elusive, and still most investors, if partly, rely on their intuition to build a better decision making. The challenge here is to make an investment strategy that is consistent over a reasonably long period, with the minimum human decision on the entire process. To this end, we propose a new stock return prediction framework that we call Ranked Information Coefficient Neural Network (RIC-NN). RIC-NN is a deep learning approach and includes the following three novel ideas: (1) nonlinear multi-factor approach, (2) stopping criteria with ranked information coefficient (rank IC), and (3) deep transfer learning among multiple regions. Experimental comparison with the stocks in the Morgan Stanley Capital International (MSCI) indices shows that RIC-NN outperforms not only off-the-shelf machine learning methods but also the average return of major equity investment funds in the last fourteen years.

Abstract--As deep learning classifiers continue to mature, model providers with sufficient data and computation resour ces are exploring approaches to monetize the development of inc reas-ingly powerful models. Licensing models is a promising appr oach, but requires a robust tool for owners to claim ownership of models, i.e. a watermark. Unfortunately, current watermarks are all vulnerable to piracy attacks, where attackers embed for ged watermarks into a model to dispute ownership. We believe properties of persistence and piracy resistance are critical to watermarks, but are fundamentally at odds with t he current way models are trained and tuned. In this work, we propose two new training techniques (out-of-bound values a nd null-embedding) that provide persistence and limit the tra ining of certain inputs into trained models. We then introduce wonder filters, a new primitive that embeds a persistent bit-sequence into a model, but only at initial training time. Wonder filter s enable model owners to embed a bit-sequence generated from their private keys into a model at training time. Attackers c annot remove wonder filters via tuning, and cannot add their own filters to pretrained models. We provide analytical proofs o f key properties, and experimentally validate them over a var iety of tasks and models. Finally, we explore a number of adaptive countermeasures, and show our watermark remains robust. Building deep neural networks (DNNs) is an expensive process. It requires significant resources, both in terms of extremely large training datasets and powerful computing resources. For example, Googles InceptionV3 model, first proposed in 2015, is based on a sophisticated architecture w ith 48 layers, trained on 1.28M labeled images over 2 weeks on 8 GPUs. As a result, model training is increasingly limited to a small group of companies with sufficient access to both data and computation. As the costs of these models continue to rise, model providers are exploring multiple approaches to monetize mo d-els to recoup their training costs. These include Machine Learning as a Service (MLaaS) platforms ( e.g. Both approaches have serious limitations. Hosted models are vulnerable to a number of model inversion or inference attacks ( e.g. Ideally, DNN watermarks are capable of providing the proof of model ownership necessary for model licensing. Upon demand, a robust watermark would provide a persistent and verifiable link between the model (or any derivatives) and it s owner. Such a watermark would require three properties.

Data loading can dominate deep neural network training time on large-scale systems. We present a comprehensive study on accelerating data loading performance in large-scale distributed training. We first identify performance and scalability issues in current data loading implementations. We then propose optimizations that utilize CPU resources to the data loader design. We use an analytical model to characterize the impact of data loading on the overall training time and establish the performance trend as we scale up distributed training. Our model suggests that I/O rate limits the scalability of distributed training, which inspires us to design a locality-aware data loading method. By utilizing software caches, our method can drastically reduce the data loading communication volume in comparison with the original data loading implementation. Finally, we evaluate the proposed optimizations with various experiments. We achieved more than 30x speedup in data loading using 256 nodes with 1,024 learners.

Neural networks are increasingly used for graph classification in a variety of contexts. Social media is a critical application area in this space, however the characteristics of social media graphs differ from those seen in most popular benchmark datasets. Social networks tend to be large and sparse, while benchmarks are small and dense. Classically, large and sparse networks are analyzed by studying the distribution of local properties. Inspired by this, we introduce Graph-Hist: an end-to-end architecture that extracts a graph's latent local features, bins nodes together along 1-D cross sections of the feature space, and classifies the graph based on this multi-channel histogram. We show that Graph-Hist improves state of the art performance on true social media benchmark datasets, while still performing well on other benchmarks. Finally, we demonstrate Graph-Hist's performance by conducting bot detection in social media. While sophisticated bot and cyborg accounts increasingly evade traditional detection methods, they leave artificial artifacts in their conversational graph that are detected through graph classification. We apply Graph-Hist to classify these conversational graphs. In the process, we confirm that social media graphs are different than most baselines and that Graph-Hist outperforms existing bot-detection models.

A BSTRACT We study the important and challenging problem of controllable generation of long-term sequential behaviors. Solutions to this problem would impact many applications, such as calibrating behaviors of AI agents in games or predicting player trajectories in sports. In contrast to the well-studied areas of controllable generation of images, text, and speech, there are significant challenges that are unique to or exacerbated by generating long-term behaviors: how should we specify the factors of variation to control, and how can we ensure that the generated temporal behavior faithfully demonstrates diverse styles? In this paper, we leverage large amounts of raw behavioral data to learn policies that can be calibrated to generate a diverse range of behavior styles (e.g., aggressive versus passive play in sports). Inspired by recent work on leveraging programmatic labeling functions, we present a novel framework that combines imitation learning with data programming to learn style-calibratable policies. Our primary technical contribution is a formal notion of style-consistency as a learning objective, and its integration with conventional imitation learning approaches. The tracking data is often obtained from multiple experts and can exhibit very diverse styles (e.g., aggressive versus passive play in sports). Our work is motivated by the opportunity to maximally leverage these datasets by cleanly extracting such styles in addition to modeling the raw behaviors.

Concept drift in learning and classification occurs when the statistical properties of either the data features or target change over time; evidence of drift has appeared in search data, medical research, malware, web data, and video. Drift adaptation has not yet been addressed in high dimensional, noisy, low-context data such as streaming text, video, or images due to the unique challenges these domains present. We present a two-fold approach to deal with concept drift in these domains: a density-based clustering approach to deal with virtual concept drift (change in statistical properties of features) and a weak-supervision step to deal with real concept drift (change in statistical properties of target). Our density-based clustering avoids problems posed by the curse of dimensionality to create an evolving 'map' of the live data space, thereby addressing virtual drift in features. Our weak-supervision step leverages high-confidence labels (oracle or heuristic labels) to generate weighted training sets to generalize and update existing deep learners to adapt to changing decision boundaries (real drift) and create new deep learners for unseen regions of the data space. Our results show that our two-fold approach performs well with >90% precision in 2018, four years after initial deployment in 2014, without any human intervention.

On the other hand, few-shot learning methods are highly vulnerable to adversarial exam ples. The goal of our work is to produce networks which both perform well at few-sh ot tasks and are simultaneously robust to adversarial examples. W e adapt ad versarial training for meta-learning, we adapt robust architectural features to s mall networks for meta-learning, we test pre-processing defenses as an alternativ e to adversarial training for meta-learning, and we investigate the advantages of rob ust meta-learning over robust transfer-learning for few-shot tasks. This work pro vides a thorough analysis of adversarially robust methods in the context of meta-lear ning, and we lay the foundation for future work on defenses for few-shot tasks. Conventional adversarial train ing and pre-processing defenses aim to produce networks that resist attack (Madry et al., 2017; Zhang e t al., 2019; Samangouei et al., 2018), but such defenses rely heavily on the availability of large t raining datasets. In applications that require few-shot learning, such as face recognition from few images, recognition of a v ideo source from a single clip, or recognition of a new object from few exa mple photos, the conventional robust training pipeline breaks down.

Classifiers are among the most widely used supervised machine learning algorithms. Many classification models exist, and choosing the right one for a given task is difficult. During model selection and debugging, data scientists need to asses classifier performance, evaluate the training behavior over time, and compare different models. Typically, this analysis is based on single-number performance measures such as accuracy. A more detailed evaluation of classifiers is possible by inspecting class errors. The confusion matrix is an established way for visualizing these class errors, but it was not designed with temporal or comparative analysis in mind. More generally, established performance analysis systems do not allow a combined temporal and comparative analysis of class-level information. To address this issue, we propose ConfusionFlow, an interactive, comparative visualization tool that combines the benefits of class confusion matrices with the visualization of performance characteristics over time. ConfusionFlow is model-agnostic and can be used to compare performances for different model types, model architectures, and/or training and test datasets. We demonstrate the usefulness of ConfusionFlow in the context of two practical problems: an analysis of the influence of network pruning on model errors, and a case study on instance selection strategies in active learning.

Progress of machine learning in critical care has been difficult to track, in part due to absence of public benchmarks. Other fields of research (such as vision and NLP) have already established various competitions and benchmarks, whereas only recent availability of large clinical datasets has enabled the possibility of public benchmarks. Taking advantage of this opportunity, we propose a public benchmark suite to address four areas of critical care, namely mortality prediction, estimation of length of stay, patient phenotyping and risk of decompensation. We define each task and compare the performance of both clinical models as well as baseline and deep models using eICU critical care dataset of around 73,000 patients. Furthermore, we investigate the impact of numerical variables as well as handling of categorical variables for each of the defined tasks.

In this paper, we address a fundamental research question of significant practical interest: Can certain theoretical characteristics of CNN architectures indicate a priori (i.e., without training) which models with highly different number of parameters and layers achieve a similar generalization performance? To answer this question, we model CNNs from a network science perspective and introduce a new, theoretically-grounded, architecture-level metric called NN-Mass. We also integrate, for the first time, the PAC-Bayes theory of generalization with small-world networks to discover new synergies among our proposed NN-Mass metric, architecture characteristics, and model generalization. With experiments on real datasets such as CIFAR-10/100, we provide extensive empirical evidence for our theoretical findings. Finally, we exploit these new insights for model compression and achieve up to 3x fewer parameters and FLOPS, while losing minimal accuracy (e.g., 96.82% vs. 97%) over large CNNs on the CIFAR-10 dataset.