Emergent properties of the local geometry of neural loss landscapesStanislav Fort Surya Ganguli Stanford University Stanford, CA, USA Stanford University Stanford, CA, USA Abstract The local geometry of high dimensional neural network loss landscapes can both challenge our cherished theoretical intuitions as well as dramatically impact the practical success of neural network training. Indeed recent works have observed 4 striking local properties of neural loss landscapes on classification tasks: (1) the landscape exhibits exactly C directions of high positive curvature, where C is the number of classes; (2) gradient directions are largely confined to this extremely low dimensional subspace of positive Hessian curvature, leaving the vast majority of directions in weight space unexplored; (3) gradient descent transiently explores intermediate regions of higher positive curvature before eventually finding flatter minima; (4) training can be successful even when confined to low dimensional random affine hy-perplanes, as long as these hyperplanes intersect a Goldilocks zone of higher than average curvature. We develop a simple theoretical model of gradients and Hessians, justified by numerical experiments on architectures and datasets used in practice, that simultaneously accounts for all 4 of these surprising and seemingly unrelated properties. Our unified model provides conceptual insights into the emergence of these properties and makes connections with diverse topics in neural networks, random matrix theory, and spin glasses, including the neural tangent kernel, BBP phase transitions, and Derrida's random energy model. 1 Introduction The geometry of neural network loss landscapes and the implications of this geometry for both optimization and generalization have been subjects of intense interest in many works, ranging from studies on the lack of local minima at significantly higher loss than that of the global minimum [1, 2] to studies debating relations between the curvature of local minima and their generalization properties [3, 4, 5, 6]. Fundamentally, the neural network loss landscape is a scalar loss function over a very high D dimensional parameter space that could depend a priori in highly nontrivial ways on the very structure of real-world data itself as well as intricate properties of the neural network architecture. Moreover, the regions of this loss landscape explored by gradient descent could themselves have highly atypical geometric properties relative to randomly chosen points in the landscape.

Choosing the most adequate kernel is crucial in many Machine Learning applications. Gaussian Process is a state-of-the-art technique for regression and classification that heavily relies on a kernel function. However, in the Gaussian Process literature, kernels have usually been either ad hoc designed, selected from a predefined set, or searched for in a space of compositions of kernels which have been defined a priori. In this paper, we propose a Genetic-Programming algorithm that represents a kernel function as a tree of elementary mathematical expressions. By means of this representation, a wider set of kernels can be modeled, where potentially better solutions can be found, although new challenges also arise. The proposed algorithm is able to overcome these difficulties and find kernels that accurately model the characteristics of the data. This method has been tested in several real-world time-series extrapolation problems, improving the state-of-the-art results while reducing the complexity of the kernels.

Code summarization (CS) and code generation (CG) are two crucial tasks in the field of automatic software development. Various neural network-based approaches are proposed to solve these two tasks separately. However, there exists a specific intuitive correlation between CS and CG, which have not been exploited in previous work. In this paper, we apply the relations between two tasks to improve the performance of both tasks. In other words, exploiting the duality between the two tasks, we propose a dual training framework to train the two tasks simultaneously. In this framework, we consider the dualities on probability and attention weights, and design corresponding regularization terms to constrain the duality. We evaluate our approach on two datasets collected from GitHub, and experimental results show that our dual framework can improve the performance of CS and CG tasks over baselines.

--Angry Birds is a popular video game in which the player is provided with a sequence of birds to shoot from a slingshot. The task of the game is to destroy all green pigs with maximum possible score. Angry Birds appears to be a difficult task to solve for artificially intelligent agents due to the sequential decision-making, non-deterministic game environment, enormous state and action spaces and requirement to differentiate between multiple birds, their abilities and optimum tapping times. We describe the application of Deep Reinforcement learning by implementing Double Dueling Deep Q-network to play Angry Birds game. One of our main goals was to build an agent that is able to compete with previous participants and humans on the first 21 levels. In order to do so, we have collected a dataset of game frames that we used to train our agent on. We evaluate our agent using results of the previous participants of AIBirds competition, results of volunteer human players and present the results of AIBirds 2018 competition. I NTRODUCTION Angry Birds has been one of the most popular video games for a period of several years. The main goal of the game is to kill all green pigs on the level together with applying as much damage as possible to the surrounding structures.

Actor and equality campaigner Geena Davis has announced that Disney has adopted a digital tool that will analyse scripts and identify opportunities to rectify any gender and ethnic biases. Davis, founder of the Geena Davis Institute on Gender in Media, was speaking at the Power of Inclusion event in New Zealand, where she outlined the development of GD-IQ: Spellcheck for Bias, a machine learning tool described as "an intervention tool to infuse diversity and inclusion in entertainment and media". Developed by the University of Southern California Viterbi School of Engineering, the Spellcheck for Bias is designed to analyse a script and determine the percentages of characters' "gender, race, LGBTQIA [and] disabilities". It can also track the percentage of "non-gender-defined speaking characters". Davis said that Disney had partnered with her institute to pilot the project: "We're going to collaborate with Disney over the next year using this tool to help their decision-making [and] identify opportunities to increase diversity and inclusion in the manuscripts that they receive. We're very excited about the possibilities with this new technology and we encourage everybody to get in touch with us and give it a try."

You might not be aware of it, but there's a quiet arms race going on over our collective reality. The fight is between those who want to subvert it and usher in a world where we no longer believe what we see on our screens and those who want to help preserve the status quo. Up until this point in time, we have largely trusted our eyes and ears when consuming audio and visual media content, but new technological systems that create something known as deepfakes, are changing that. And as these deepfake videos nudge into the mainstream, experts are increasingly worried about the ramifications it will have on the information sharing that underpins society. Dr Richard Nock is the head of machine learning at CSIRO's Data 61 and understands the daunting potential of the technology that powers deepfake videos.

We propose multivariate nonstationary Gaussian processes for jointly modeling multiple clinical variables, where the key parameters, length-scales, standard deviations and the correlations between the observed output, are all time dependent. We perform posterior inference via Hamiltonian Monte Carlo (HMC). We also provide methods for obtaining computationally efficient gradient-based maximum a posteriori (MAP) estimates. We validate our model on synthetic data as well as on electronic health records (EHR) data from Kaiser Permanente (KP). We show that the proposed model provides better predictive performance over a stationary model as well as uncovers interesting latent correlation processes across vitals which are potentially predictive of patient risk.

Chris discusses how rule-based systems are moving to learning-based systems in various enterprise use cases. Sramana Mitra: Let's start by having you introduce yourself and Genesys and what work you're doing around artificial intelligence. Christopher Connolly: I am the Vice President of our Solutions Strategy Group. Genesys is the number one customer experience platform. It enables companies to create exceptional omni-channel experiences and relationships.

"It's been a surprise that the Russians, the GRU, this unit, have felt free to go ahead and carry out this extreme malign activity … That's been a shock," the New York Times quotes an unnamed European security official as saying. Western intelligence agencies only recently became aware of this Russian covert operations unit, according to reports. This is despite Unit 29155 agents engaging in espionage activities for more than a decade. But the pieces have begun to fall into place. And the evidence reveals a Kremlin campaign to convince its people that their troubled nation is back on the path to "greatness" -- all while undermining the Western liberal democratic notion of "rules-based order".

Hummingbird Technologies are a world-leading AI and machine learning business in the crop analysis space. We consolidate data from drones, planes and satellites and deliver value-driven actionable insights for farmers, agronomists and food companies. The Hummingbird platform delivers greater insight into crop health and yield potential with a range of crop-specific AI tools for earlier disease identification, optimum nutrient management, detailed plant counting, crop development modelling and yield prediction. Backed by Sir James Dyson, the European Space Agency, BASF and some of the leading tech VC and large agro businesses, Hummingbird have operations in UK, Brazil, Russia, Ukraine, Australia and North America.