Deep Learning
NVIDIA and HPE hop on board the AI accessibility train
AI and augmented reality are set to create a digital disruption in the delivery industry, according to an Accenture report. Driverless delivery vehicles and robots are among the technologies set to facilitate the link from products and services to customers. But the post and parcel industry is not the first to see sweeping changes from these technologies, and tech companies are responding with better tools for development and adoption. NVIDIA, for example, is offering a cloud-based container registry for users of the Amazon Elastic Compute Cloud P3 instances, according to a company announcement Wednesday. The no-cost tool offers developers tools for democratized AI, such as powerful GPUs through the cloud and a performance-tuned deep learning software stack.
A Visual Guide to Evolution Strategies
In this post I explain how evolution strategies (ES) work with the aid of a few visual examples. I try to keep the equations light, and I provide links to original articles if the reader wishes to understand more details. This is the first post in a series of articles, where I plan to show how to apply these algorithms to a range of tasks from MNIST, OpanAI Gym, Roboschool to PyBullet environments. Neural network models are highly expressive and flexible, and if we are able to find a suitable set of model parameters, we can use neural nets to solve many challenging problems. Deep learning's success largely comes from the ability to use the backpropagation algorithm to efficiently calculate the gradient of an objective function over each model parameter. With these gradients, we can efficiently search over the parameter space to find a solution that is often good enough for our neural net to accomplish difficult tasks. However, there are many problems where the backpropagation problem cannot be used.
Top 5 Deep Learning and AI Stories - October 20, 2017
Forbes: How AI can transform businesses 2. GigaOm's "Voices in AI" podcast features AI luminaries 3. NVIDIA's Inception Program for AI startups adds 2,000th member 4. NVIDIA teaches the world about deep learning in finance through workshops 5. What AI can accomplish right now – and the silicon powering it all 5. HOW ARTIFICIAL INTELLIGENCE CAN TRANSFORM BUSINESSES A question originally asked on Quora, Forbes shares the leading answer by Tony Paikeday, product marketing director at NVIDIA. After providing several examples of businesses across industries who are already using AI, he shares the quickest method to adopting AI: "The fastest way to accelerate AI for business is leveraging powerful and energy-efficient GPUs. READ ARTICLE 6. GIGAOM'S "VOICES IN AI" PODCAST FEATURES AI LUMINARIES GigaOm's "Voices in AI" podcast debuted this month, featuring episodes with leading researchers and AI luminaries – including NVIDIA's Bryan Catanzaro. In episode 13, Bryan focuses on AI and the future of work: "I like to think about artificial intelligence as making tools that can perform intellectual work. LISTEN TO PODCAST 7. NVIDIA'S INCEPTION PROGRAM FOR AI STARTUPS ADDS 2,000TH MEMBER Less than 18 months after its launch, NVIDIA's Inception program -- which helps accelerate startups pushing the frontiers of AI and data science -- has signed up its 2,000th member company.
Could We Build a Machine with Consciousness?
Not quite yet, but neuroscience research is giving us some clues about how it may be possible in the not-too-distant future. In a paper published in Science today, a trio of neuroscientists, led by Stanislas Dehaene from Colle ge de France in Paris, try to pin down exactly what we mean by "consciousness" in order to work out whether machines could ever possess it. As they see it, there are three kinds of consciousness--and computers have so far mastered only one of them. One is subconsciousness, the huge range of processes in the brain where most human intelligence lies. That's what powers our ability to, say, determine a chess move or spot a face without really knowing how we did it. That, the researchers say, is broadly comparable to the kind of processing that modern-day AIs, such as DeepMind's AlphaGo or Face's facial recognition algorithms, are good at.
Elon Musk's artificial intelligence company created virtual robots that can sumo wrestle and play soccer
Elon Musk's artificial intelligence company created virtual robots that can sumo wrestle and play soccer. Following is a transcript of the video. These AI robots are getting physical. They may look goofy but they're smarter than you think. OpenAI's bots can teach themselves how to sumo wrestle and play soccer.
Machine Learning Offers a Path to Deeper Insight
Data scientists, developers, and researchers are using machine learning to gain insights previously out of reach. Programs that learn from experience are helping them discover how the human genome works, understand consumer behavior to a degree never before possible, and build systems for purchase recommendations, image recognition, and fraud prevention, among other uses. Now you can scale your machine learning and deep learning applications quickly – and gain insights more efficiently – with your existing hardware infrastructure. Popular open frameworks newly optimized for Intel, together with our advanced math libraries, make Intel Architecture-based platforms a smart choice for these projects.
Today's Weak AI Lacks Intelligence
While Deep Learning and other ANN-based methods of machine learning have produced some amazing capabilities over the past decade, they still leave me wanting more intelligence than they can deliver. The "point neuron" used in ANN is based on an understanding of neuroscience we had back in the 1970s. Just in the past couple of decades, we have learned more about the pyramidal neuron in the neocortex than we have ever known before. This knowledge has never been applied to the old ANN point neuron. In fact, even some Deep Learning experts like Oriol Vinyals of DeepMind says that Deep Learning is not AI.
Learning neural trans-dimensional random field language models with noise-contrastive estimation
Trans-dimensional random field language models (TRF LMs) where sentences are modeled as a collection of random fields, have shown close performance with LSTM LMs in speech recognition and are computationally more efficient in inference. However, the training efficiency of neural TRF LMs is not satisfactory, which limits the scalability of TRF LMs on large training corpus. In this paper, several techniques on both model formulation and parameter estimation are proposed to improve the training efficiency and the performance of neural TRF LMs. First, TRFs are reformulated in the form of exponential tilting of a reference distribution. Second, noise-contrastive estimation (NCE) is introduced to jointly estimate the model parameters and normalization constants. Third, we extend the neural TRF LMs by marrying the deep convolutional neural network (CNN) and the bidirectional LSTM into the potential function to extract the deep hierarchical features and bidirectionally sequential features. Utilizing all the above techniques enables the successful and efficient training of neural TRF LMs on a 40x larger training set with only 1/3 training time and further reduces the WER with relative reduction of 4.7% on top of a strong LSTM LM baseline.
Contextual Regression: An Accurate and Conveniently Interpretable Nonlinear Model for Mining Discovery from Scientific Data
Machine learning algorithms such as linear regression, SVM and neural network have played an increasingly important role in the process of scientific discovery. However, none of them is both interpretable and accurate on nonlinear datasets. Here we present contextual regression, a method that joins these two desirable properties together using a hybrid architecture of neural network embedding and dot product layer. We demonstrate its high prediction accuracy and sensitivity through the task of predictive feature selection on a simulated dataset and the application of predicting open chromatin sites in the human genome. On the simulated data, our method achieved high fidelity recovery of feature contributions under random noise levels up to 200%. On the open chromatin dataset, the application of our method not only outperformed the state of the art method in terms of accuracy, but also unveiled two previously unfound open chromatin related histone marks. Our method can fill the blank of accurate and interpretable nonlinear modeling in scientific data mining tasks.
Regularization for Deep Learning: A Taxonomy
Kukačka, Jan, Golkov, Vladimir, Cremers, Daniel
Regularization is one of the crucial ingredients of deep learning, yet the term regularization has various definitions, and regularization methods are often studied separately from each other. In our work we present a systematic, unifying taxonomy to categorize existing methods. We distinguish methods that affect data, network architectures, error terms, regularization terms, and optimization procedures. We do not provide all details about the listed methods; instead, we present an overview of how the methods can be sorted into meaningful categories and sub-categories. This helps revealing links and fundamental similarities between them. Finally, we include practical recommendations both for users and for developers of new regularization methods.