The scientific literature is a rich source of information for data mining with conceptual knowledge graphs; the open science movement has enriched this literature with complementary source code that implements scientific models. To exploit this new resource, we construct a knowledge graph using unsupervised learning methods to identify conceptual entities. We associate source code entities to these natural language concepts using word embedding and clustering techniques. Practical naming conventions for methods and functions tend to reflect the concept(s) they implement. We take advantage of this specificity by presenting a novel process for joint clustering text concepts that combines word-embeddings, nonlinear dimensionality reduction, and clustering techniques to assist in understanding, organizing, and comparing software in the open science ecosystem. With our pipeline, we aim to assist scientists in building on existing models in their discipline when making novel models for new phenomena. By combining source code and conceptual information, our knowledge graph enhances corpus-wide understanding of scientific literature.

Principal component analysis, dictionary learning, and auto-encoders are all unsupervised methods for learning representations from a large amount of training data. In all these methods, the higher the dimensions of the input data, the longer it takes to learn. We introduce a class of neural networks, termed RandNet, for learning representations using compressed random measurements of data of interest, such as images. RandNet extends the convolutional recurrent sparse auto-encoder architecture to dense networks and, more importantly, to the case when the input data are compressed random measurements of the original data. Compressing the input data makes it possible to fit a larger number of batches in memory during training. Moreover, in the case of sparse measurements,training is more efficient computationally. We demonstrate that, in unsupervised settings, RandNet performs dictionary learning using compressed data. In supervised settings, we show that RandNet can classify MNIST images with minimal loss in accuracy, despite being trained with random projections of the images that result in a 50% reduction in size. Overall, our results provide a general principled framework for training neural networks using compressed data.

The success of Deep Learning methods is not well understood, though various attempts at explaining it have been made, typically centered on properties of stochastic gradient descent. Even less clear is why certain neural network architectures perform better than others. We provide a potential opening with the hypothesis that neural network training is a form of Genetic Programming.

Crazyhouse is a chess variant that incorporates all of the classical chess rules, but allows users to drop pieces captured from the opponent as a normal move. Until 2018, all competitive computer engines for this board game made use of an alpha-beta pruning algorithm with a hand-crafted evaluation function for each position. Previous machine learning-based algorithms for just regular chess, such as NeuroChess and Giraffe, took hand-crafted evaluation features as input rather than a raw board representation. More recent projects, such as AlphaZero, reached massive success but required massive computational resources in order to reach its final strength. This paper describes the development of SixtyFour, an engine designed to compete in the chess variant of Crazyhouse with limited hardware. This specific variant poses a multitude of significant challenges due to its large branching factor, state-space complexity, and the multiple move types a player can make. We propose the novel creation of a neural network-based evaluation function for Crazyhouse. More importantly, we evaluate the effectiveness of an ensemble model, which allows the training time and datasets to be easily distributed on regular CPU hardware commodity. Early versions of the network have attained a playing level comparable to a strong amateur on online servers.

The US remains the global leader in artificial intelligence (AI) innovation, despite big moves by China to change its standing, according to a study from the Center for Data Innovation, released this week. The study focused on six distinct categories--talent, research, development, adoption, data and hardware--and found that the US was far ahead of both China and the European Union in at least four of them. With the help of Amazon, Apple, Facebook, Google, Intel and Microsoft, the US was able to financially dominate the last wave of digital innovation. But China has doubled down on its efforts to become equals with the US and now invests heavily into educating and funding AI research. SEE: IT leader's guide to deep learning (Tech Pro Research) "The United States is leading in AI today, but it should not rest on its laurels," Michael McLaughlin, the Center's research analyst and the report's lead author, wrote in the study.

To enable wider use of this powerful deep learning architecture, we propose two new methods. The first, adaptive attention span is a way to make Transformer networks more efficient for longer sentences. With this method, we were able to increase the attention span of a Transformer to over 8,000 tokens without significantly increasing computation time or memory footprint. The second, all-attention layer is a way to simplify the model architecture of Transformer networks. Even with a much simpler architecture, our all-attention network matched the state-of-the-art performance of Transformer networks.

After six months of competition (and a few last-minute submissions), we are happy to announce the conclusion and winners of the Obstacle Tower Challenge. We want to thank all of the participants for both rounds and congratulate Alex Nichol, the Compscience.org We are also excited to share that we have open-sourced Obstacle Tower for the research community to extend for their own needs. We started this challenge in February as a way to help foster research in the AI community, by providing a challenging new benchmark of agent performance built in Unity, which we called Obstacle Tower. The Obstacle Tower was developed to be difficult for current machine learning algorithms to solve, and push the boundaries of what was possible in the field by focusing on procedural generation. Key to that was only allowing participants access to one hundred instances of the Obstacle Tower, and evaluating their trained agents on a set of unique procedurally generated towers they had never seen before.

The most common way to train a neural network today is by using gradient descent or one of its variants like Adam. Gradient descent is an iterative optimization algorithm for finding the minimum of a function. Simply put, in optimization problems, we are interested in some metric P and we want to find a function (or parameters of a function) that maximizes (or minimizes) this metric on some data (or distribution) D. This sounds just like Machine (or Deep) Learning. We have some metric, like accuracy, or even better precision/recall or F1 score, we have a model with learnable parameters (our network) and we have our data (the training and test sets). Using gradient descent we are "searching' or "optimizing" our model's parameters in a way that will eventually maximize our metric (accuracy) on our data, both on the training and the test sets.

Tech decision makers are (and should keep) looking for ways to effectively implement artificial intelligence technologies into their businesses and, therefore, drive value. And though all AI technologies most definitely have their own merits, not all of them are worth investing in. If one thing and only one thing happens after you read this article, we hope it is that you are inspired to join the 62% of companies who boosted their enterprises in 2018 by adopting Artificial Intelligence into their workflow. Natural language generation is an AI sub-discipline that converts data into text, enabling computers to communicate ideas with perfect accuracy. It is used in customer service to generate reports and market summaries and is offered by companies like Attivio, Automated Insights, Cambridge Semantics, Digital Reasoning, Lucidworks, Narrative Science, SAS, and Yseop.

The Linux Foundation has registered trademarks and uses trademarks. For a list of trademarks of The Linux Foundation, please see our Trademark Usage page. Linux is a registered trademark of Linus Torvalds.