Training machine learning (ML) models is expensive in terms of computational power, large amounts of labeled data, and human expertise. Thus, ML models constitute intellectual property (IP) and business value for their owners. Embedding digital watermarks during model training allows a model owner to later identify their models in case of theft or misuse. However, model functionality can also be stolen via model extraction, where an adversary trains a surrogate model using results returned from a prediction API of the original model. Recent work has shown that model extraction is a realistic threat. Existing watermarking schemes are ineffective against IP theft via model extraction since it is the adversary who trains the surrogate model. In this paper, we introduce DAWN (Dynamic Adversarial Watermarking of Neural Networks), the first approach to use watermarking to deter IP theft via model extraction. Unlike prior watermarking schemes, DAWN does not impose changes to the training process. Instead, it operates at the prediction API of the protected model, by dynamically changing the responses for a small subset of queries (e.g. $<0.5\%$) from API clients. This set represents a watermark that will be embedded in case a client uses its queries to train a surrogate model. We show that DAWN is resilient against two state-of-the-art model extraction attacks, effectively watermarking all extracted surrogate models, allowing model owners to reliably demonstrate ownership (with confidence $>1-2^{-64}$), incurring negligible loss of prediction accuracy ($0.03-0.5\%$).

Environmental stresses such as drought and heat can cause substantial yield loss in agriculture. As such, hybrid crops which are tolerant to drought and heat stress would produce more consistent yields compared to the hybrids which are not tolerant to these stresses. In the 2019 Syngenta Crop Challenge, Syngenta released several large datasets that recorded the yield performances of 2,452 corn hybrids planted in 1,560 locations between 2008 and 2017 and asked participants to classify the corn hybrids as either tolerant or susceptible to drought stress, heat stress, and combined drought and heat stress. As one of the winning teams, we designed a two-step approach to solve this problem in an unsupervised way since no data was provided that classified any set of hybrids as tolerant or susceptible to any type of stress. First, we designed a deep convolutional neural network (CNN) that took advantage of state-of-the-art modeling and solution techniques to extract stress metrics for each type of stress. Our CNN model was found to successfully distinguish between the low and high stress environments due to considering multiple factors such as planting/harvest dates, daily weather, and soil conditions. Then, we conducted a linear regression of the yield of hybrid against each stress metric, and classified the hybrid based on the slope of the regression line, since the slope of the regression line showed how sensitive a hybrid was to a specific environmental stress. Our results suggested that only 14 % of the corn hybrids were tolerant to at least one type of stress.

It has been established that diverse behaviors spanning the controllable subspace of an Markov decision process can be trained by rewarding a policy for being distinguishable from other policies \citep{gregor2016variational, eysenbach2018diversity, warde2018unsupervised}. However, one limitation of this formulation is generalizing behaviors beyond the finite set being explicitly learned, as is needed for use on subsequent tasks. Successor features \citep{dayan93improving, barreto2017successor} provide an appealing solution to this generalization problem, but require defining the reward function as linear in some grounded feature space. In this paper, we show that these two techniques can be combined, and that each method solves the other's primary limitation. To do so we introduce Variational Intrinsic Successor FeatuRes (VISR), a novel algorithm which learns controllable features that can be leveraged to provide enhanced generalization and fast task inference through the successor feature framework. We empirically validate VISR on the full Atari suite, in a novel setup wherein the rewards are only exposed briefly after a long unsupervised phase. Achieving human-level performance on 14 games and beating all baselines, we believe VISR represents a step towards agents that rapidly learn from limited feedback.

Recently, substantial progress has been made regarding theoretical analysis of the generalization of deep learning models [see Zhang et al., 2016, Dziugaite and Roy, 2017, Bartlett et al., 2017, Neyshabur et al., 2017, 2018, Arora et al., 2018, Neyshabur et al., 2019]. One interesting point that has been explored, with roots in [Bartlett, 1998], is that even if there are many parameters, the set of models computable using weights with small magnitude is limited enough to provide leverage for induction [Bartlett et al., 2017, Neyshabur et al., 2018]. Intuitively, if the weights start small, since the most popular training algorithms make small, incremental updates that get smaller as the training accuracy improves, there is a tendency for these algorithms to produce small weights.

We propose an efficient neural framework for sentence-level discourse analysis in accordance with Rhetorical Structure Theory (RST). Our framework comprises a discourse segmenter to identify the elementary discourse units (EDU) in a text, and a discourse parser that constructs a discourse tree in a top-down fashion. Both the segmenter and the parser are based on Pointer Networks and operate in linear time. Our segmenter yields an $F_1$ score of 95.4, and our parser achieves an $F_1$ score of 81.7 on the aggregated labeled (relation) metric, surpassing previous approaches by a good margin and approaching human agreement on both tasks (98.3 and 83.0 $F_1$).

One of my favorite things about deep reinforcement learning is that, unlike supervised learning, it really, really doesn't want to work. Throwing a neural net at a computer vision problem might get you 80% of the way there. Throwing a neural net at an RL problem will probably blow something up in front of your face -- and it will blow up in a different way each time you try. A lot of the biggest challenges in RL revolve around two questions: how we interact with the environment effectively (e.g. In this post, I want to explore a few recent directions in deep RL research that attempt to address these challenges, and do so with particularly elegant parallels to human cognition. This post will begin with a quick review of two canonical deep RL algorithms -- DQN and A3C -- to provide us some intuitions to refer back to, and then jump into a deep dive on a few recent papers and breakthroughs in the categories described above.

IMAGE: This is a schematic illustration of MEGNet models. Nanoengineers at the University of California San Diego have developed new deep learning models that can accurately predict the properties of molecules and crystals. By enabling almost instantaneous property predictions, these deep learning models provide researchers the means to rapidly scan the nearly-infinite universe of compounds to discover potentially transformative materials for various technological applications, such as high-energy-density Li-ion batteries, warm-white LEDs, and better photovoltaics. To construct their models, a team led by nanoengineering professor Shyue Ping Ong at the UC San Diego Jacobs School of Engineering used a new deep learning framework called graph networks, developed by Google DeepMind, the brains behind AlphaGo and AlphaZero. Graph networks have the potential to expand the capabilities of existing AI technology to perform complicated learning and reasoning tasks with limited experience and knowledge--something that humans are good at.

While supervised learning has tremendously improved AI performance in image classification, a major drawback is its reliance on large-scale labeled datasets. This has prompted researchers to explore the potential of unsupervised learning and semi-supervised learning -- techniques that forego data annotation but have their own drawback: diminished accuracy. A new paper from Google's UK-based research company DeepMind addresses this with a model based on Contrastive Predictive Coding (CPC) that outperforms the fully-supervised AlexNet model in Top-1 and Top-5 accuracy on ImageNet. CPC was introduced by DeepMind in 2018. The unsupervised learning approach uses a powerful autoregressive model to extract representations of high-dimensional data to predict future samples.

Noelia Jiménez Martínez is Head of Data Science and Astrophysics at Unbound. She holds a PhD in Numerical Astrophysics from the UNLP (La Plata, Argentina) applied to Galaxy Formation and Chemical Evolution. Before transitioning from Academia, she was an Astrophysics Researcher at the University of St Andrews (Scotland), and previously had several postdocs positions in different universities across Europe that gave her the chance to collaborate with a huge diversity of people from several fields and backgrounds. Prior to joining Unbound she worked as a Data Science Consultant (Pivigo) in London, where she managed several data science teams working with big-to-small companies (Barclays, Criteo, Royal Mail, startups, etc) and lectured academics transitioning to Industry in the S2DS (science to data science school) in fields as Machine Learning, Statistics and Deep Learning. She is also the author of a book exploring/building empathy and social skills among academics from'hard' sciences: 'Data: A Guide to Humans'.

Reproducibility puts the science in the computer science of AI. It's how researchers can prove their AI systems are robust and reliable. To support reproducibility for AI models, Facebook today released PyTorch Hub in beta, an API and workflow for research reproducibility and support. PyTorch Hub can quickly publish pretrained models to a GitHub repository by adding a hubconf.py PyTorch Hub comes with support for models in Google Colab and PapersWithCode.