In contrast with previous models: (i) we train a Artificial Neural Networks, and Long Short-Term vanilla char-LSTM on a more realistic variety and Memory Networks more specifically, have consistently amount of data, focusing on a limited amount of demonstrated great capabilities in the area child-directed language; (ii) we do not rely on extrinsic of language modeling. In addition to generating evaluations or downstream tasks, instead we credible surface patterns, they show excellent performances introduce a methodology to evaluate how the distribution when tested on very specific grammatical of grammatical items, over time, comes abilities (Gulordava et al., 2018; Lakretz et al., to approximate the one in the input, through a continuous 2019), without requiring any prior bias towards the process and (iii) we tentatively explore the syntactic structure of natural languages.

Relatedness-based evaluation - known as intrinsic evaluation in the context of embedding-based A standard approach to evaluate a relatednessbased models - requires the construction of a dataset of model is the comparison of the semantic human annotations, which may be collected via ranking it produces with the corresponding ranking two different approaches. The former relies on a determined from human annotations. However, small group of linguistic experts to create a gold the relevance of rank mismatches may depend standard dataset, which is reliable but very expensive on the involved positions; in particular, top ranks and, due to the subjectivity of relatedness and are considered more important in many contexts, to the limited number of annotations, highly susceptible two prominent examples being content-based recommenders to bias and lack of statistical significance (De Gemmis et al., 2008, 2015; Lops (Blanco et al., 2013; Faruqui et al., 2016). The latter et al., 2011; Mladenic, 1999) and semantic matching relies on a large group of non-experts, typically (Giunchiglia et al., 2004; Li and Xu, 2014; associated with a crowdsourcing service (e.g., Amazon Wan et al., 2016). The greater significance of top MTurk, ProlificAcademic, SocialSci, Crowd-ranks compared with low ranks is actually a pretty Flower, ClickWorker, CrowdSource), it is typically common phenomenon, as it can be argued from more affordable, and it has been proven to be repeatable the attempts to overweight the former in the context and reliable (Blanco et al., 2013). of ranking correlation (Blest, 2000; Pinto da In the next sections we describe and justify a Costa and Soares, 2005; Dancelli et al., 2013; Iman protocol to construct a dataset based on semantic and Conover, 1987; Maturi and Abdelfattah, 2008; relatedness between pairs of tokens

Neural architecture search (NAS) has attracted a lot of attention and has been illustrated to bring tangible benefits in a large number of applications in the past few years. Network topology and network size have been regarded as two of the most important aspects for the performance of deep learning models and the community has spawned lots of searching algorithms for both aspects of the neural architectures. However, the performance gain from these searching algorithms is achieved under different search spaces and training setups. This makes the overall performance of the algorithms to some extent incomparable and the improvement from a sub-module of the searching model unclear. In this paper, we propose NATS-Bench, a unified benchmark on searching for both topology and size, for (almost) any up-to-date NAS algorithm. NATS-Bench includes the search space of 15,625 neural cell candidates for architecture topology and 32,768 for architecture size on three datasets. We analyse the validity of our benchmark in terms of various criteria and performance comparison of all candidates in the search space. We also show the versatility of NATS-Bench by benchmarking 13 recent state-of-the-art NAS algorithms on it. All logs and diagnostic information trained using the same setup for each candidate are provided. This facilitates a much larger community of researchers to focus on developing better NAS algorithms in a more comparable and computationally cost friendly environment. All codes are publicly available at: https://xuanyidong.com/assets/projects/NATS-Bench .

When developing models for regulated decision making, sensitive features like age, race and gender cannot be used and must be obscured from model developers to prevent bias. However, the remaining features still need to be tested for correlation with sensitive features, which can only be done with the knowledge of those features. We resolve this dilemma using a fully homomorphic encryption scheme, allowing model developers to train linear regression and logistic regression models and test them for possible bias without ever revealing the sensitive features in the clear. We demonstrate how it can be applied to leave-one-out regression testing, and show using the adult income data set that our method is practical to run.

Evolution strategies (ES), as a family of black-box optimization algorithms, recently emerge as a scalable alternative to reinforcement learning (RL) approaches such as Q-learning or policy gradient, and are much faster when many central processing units (CPUs) are available due to better parallelization. In this paper, we propose a systematic incremental learning method for ES in dynamic environments. The goal is to adjust previously learned policy to a new one incrementally whenever the environment changes. We incorporate an instance weighting mechanism with ES to facilitate its learning adaptation, while retaining scalability of ES. During parameter updating, higher weights are assigned to instances that contain more new knowledge, thus encouraging the search distribution to move towards new promising areas of parameter space. We propose two easy-to-implement metrics to calculate the weights: instance novelty and instance quality. Instance novelty measures an instance's difference from the previous optimum in the original environment, while instance quality corresponds to how well an instance performs in the new environment. The resulting algorithm, Instance Weighted Incremental Evolution Strategies (IW-IES), is verified to achieve significantly improved performance on a suite of robot navigation tasks. This paper thus introduces a family of scalable ES algorithms for RL domains that enables rapid learning adaptation to dynamic environments.

Code comment generation which aims to automatically generate natural language descriptions for source code, is a crucial task in the field of automatic software development. Traditional comment generation methods use manually-crafted templates or information retrieval (IR) techniques to generate summaries for source code. In recent years, neural network-based methods which leveraged acclaimed encoder-decoder deep learning framework to learn comment generation patterns from a large-scale parallel code corpus, have achieved impressive results. However, these emerging methods only take code-related information as input. Software reuse is common in the process of software development, meaning that comments of similar code snippets are helpful for comment generation. Inspired by the IR-based and template-based approaches, in this paper, we propose a neural comment generation approach where we use the existing comments of similar code snippets as exemplars to guide comment generation. Specifically, given a piece of code, we first use an IR technique to retrieve a similar code snippet and treat its comment as an exemplar. Then we design a novel seq2seq neural network that takes the given code, its AST, its similar code, and its exemplar as input, and leverages the information from the exemplar to assist in the target comment generation based on the semantic similarity between the source code and the similar code. We evaluate our approach on a large-scale Java corpus, which contains about 2M samples, and experimental results demonstrate that our model outperforms the state-of-the-art methods by a substantial margin.

Grounding language in contextual information is crucial for fine-grained natural language understanding. One important task that involves grounding contextual modifiers is color generation. Given a reference color "green", and a modifier "bluey", how does one generate a color that could represent "bluey green"? We propose a computational pragmatics model that formulates this color generation task as a recursive game between speakers and listeners. In our model, a pragmatic speaker reasons about the inferences that a listener would make, and thus generates a modified color that is maximally informative to help the listener recover the original referents. In this paper, we show that incorporating pragmatic information provides significant improvements in performance compared with other state-of-the-art deep learning models where pragmatic inference and flexibility in representing colors from a large continuous space are lacking. Our model has an absolute 98% increase in performance for the test cases where the reference colors are unseen during training, and an absolute 40% increase in performance for the test cases where both the reference colors and the modifiers are unseen during training.

Wilkinson recently had a close call when a shark trailed him, only inches away. Wilkinson recently had a close call when a shark trailed him, only inches away. The surfer had no idea a shark was trailing him. Near Sharpes Beach in Australia, professional surfer Matt Wilkinson was paddling on his board on Wednesday. Unbeknownst to him, a shark quickly surfaced and began stalking the surfing world champion, at one point only inches away.

Editor's note: Astrobites is a graduate-student-run organization that digests astrophysical literature for undergraduate students. We hope you enjoy this post from astrobites; the original can be viewed at astrobites.org. Galaxy clusters are among the largest gravitationally bound structures in the universe. One of their defining characteristics is that they tend to be embedded within a large reservoir of superheated gas, known as the intracluster medium (ICM). With temperatures up to 108 Kelvin, the ICM is a strong emitter of X-ray radiation.

Australia's position as one of the world leaders in artificial intelligence (AI) and machine learning will be further boosted thanks to $20 million towards a new national centre, to be based at the University of Adelaide. The Centre for Augmented Reasoning is an investment by the Australian Government in people and research to make computers better at interacting with humans, so that all technology might be easier and safer to use. The new centre builds on the expertise of the internationally regarded Australian Institute for Machine Learning (AIML) at the University of Adelaide, jointly established with the South Australian Government and based in Adelaide's Lot Fourteen innovation precinct. "The $20 million announced in this week's Federal Budget is a very exciting development, representing seed investment in our new centre. This will be a solid foundation for industry and government to build on, to ensure Australia captures the full benefits from the artificial intelligence (AI) revolution. "The new centre will be a major boost to the University of Adelaide's capabilities, and will create new jobs in research, and opportunities for students.