This paper proposes an alternating back-propagation algorithm for learning the generator network model. The model is a non-linear generalization of factor analysis. In this model, the mapping from the continuous latent factors to the observed signal is parametrized by a convolutional neural network. The alternating back-propagation algorithm iterates the following two steps: (1) Inferential back-propagation, which infers the latent factors by Langevin dynamics or gradient descent. (2) Learning back-propagation, which updates the parameters given the inferred latent factors by gradient descent. The gradient computations in both steps are powered by back-propagation, and they share most of their code in common. We show that the alternating back-propagation algorithm can learn realistic generator models of natural images, video sequences, and sounds. Moreover, it can also be used to learn from incomplete or indirect training data.

This paper presents Rudra, a parameter server based distributed computing framework tuned for training large-scale deep neural networks. Using variants of the asynchronous stochastic gradient descent algorithm we study the impact of synchronization protocol, stale gradient updates, minibatch size, learning rates, and number of learners on runtime performance and model accuracy. We introduce a new learning rate modulation strategy to counter the effect of stale gradients and propose a new synchronization protocol that can effectively bound the staleness in gradients, improve runtime performance and achieve good model accuracy. Our empirical investigation reveals a principled approach for distributed training of neural networks: the mini-batch size per learner should be reduced as more learners are added to the system to preserve the model accuracy. We validate this approach using commonly-used image classification benchmarks: CIFAR10 and ImageNet.

Only 26 percent of computer professionals were women in 2013, according to a recent review by the American Association of University Women. That figure dropped by nine percent since 1990. Some say the industry is masculine by design. Others claim computer culture is unwelcoming -- even hostile -- to women. So, while STEM fields like biology, chemistry, and engineering see an increase in diversity, computing does not.

Vatican City, Dec 4, 2016 / 03:03 am (CNA/EWTN News).- This week the Vatican hosted a high-level discussion in the world of science, gathering experts to discuss the progress, benefits and limits of advances in artificial intelligence. A new conference at the Vatican drew experts in various fields of science and technology for a two-day dialogue on the "Power and Limits of Artificial Intelligence," hosted by the Pontifical Academy for Sciences. Among the scheduled speakers were several prestigious scientists, including Stephen Hawkins, a prominent British professor at the University of Cambridge and a self-proclaimed atheist, as well as a number of major tech heads such as Demis Hassabis, CEO of Google DeepMind, and Yann LeCun of Facebook. The event, which ran from Nov. 30-Dec.

Nearly 140 private companies working to advance artificial intelligence technologies have been acquired since 2011, with over 40 acquisitions taking place in 2016 alone (as of 10/7/2016). Corporate giants like Google, IBM, Yahoo, Intel, Apple and Salesforce, are competing in the race to acquire private AI companies, with Samsung emerging as a new entrant this month with its acquisition of startup Viv Labs, which is developing a Siri-like AI assistant. Google has been the most prominent global player, with 11 acquisitions in the category under its belt (follow all of Google's M&A activity here through our real-time Google acquisitions tracker). In 2013, the corporate giant picked up deep learning and neural network startup DNNresearch from the computer science department at the University of Toronto. This acquisition reportedly helped Google make major upgrades to its image search feature.

Andrej Karpathy's post "The Unreasonable Effectiveness of Recurrent Neural Networks" made splashes last year. The basic premise is that you can create a recurrent neural network to learn language features character-by-character. But is the resultant model any different from a Markov chain built for the same purpose? I implemented a character-by-character Markov chain in R to find out. First, let's play a variation of the Imitation Game with generated text from Karpathy's tinyshakespeare dataset.

In the past few months I've been fascinated with "Deep Learning", especially its applications to language and text. I've spent the bulk of my career in financial technologies, mostly in algorithmic trading and alternative data services. You can see where this is going. I wrote this to get my ideas straight in my head. While I've become a "Deep Learning" enthusiast, I don't have too many opportunities to brain dump an idea in most of its messy glory.

We are indeed living in interesting times, where we celebrate human-built machines defeating the best human minds at variety of activities. IBM Deep Blue's win against Chess champion Gary kasparov in 1997, IBM watson acing Jeopardy in 2011 and now Google DeepMind reportedly wining'Go' with high precision, being cited as a major breakthrough in AI, which even Facebook claims their team came close to acing the game as well. DeepMind goes against the'Go' champion, to be streamed live for the world to witness. While these feats are undoubtedly remarkable, and as understandable its creating quite a buzz in the AI community; as it provides the glimpse to the future seen only in sci-fi. As exciting as it may sound, it leaves a few questions before us.

This course continues where my first course, Deep Learning in Python, left off. You already know how to build an artificial neural network in Python, and you have a plug-and-play script that you can use for TensorFlow. You learned about backpropagation (and because of that, this course contains basically NO MATH), but there were a lot of unanswered questions. How can you modify it to improve training speed? In this course you will learn about batch and stochastic gradient descent, two commonly used techniques that allow you to train on just a small sample of the data at each iteration, greatly speeding up training time.

However, there are two distinct species both referred to as "bots": The first article of this series argued that the latter have a number of serious disadvantages that make them virtually unacceptable. They face problems with deep linking, discoverability and lack of bot-to-bot communication protocols. Intelligent conversational agents offer an intriguing solution to these problems: they don't need new protocols or APIs to communicate with each other and with "master bots" such as Google Assistant. They will communicate in plain English! Bots are the new websites -- every organization will soon have its own bot.