Last week, we discussed Multi-class SVM loss; specifically, the hinge loss and squared hinge loss functions. A loss function, in the context of Machine Learning and Deep Learning, allows us to quantify how "good" or "bad" a given classification function (also called a "scoring function") is at correctly classifying data points in our dataset. In fact, if you have done previous work in Deep Learning, you have likely heard of this function before -- do the terms Softmax classifier and cross-entropy loss sound familiar? I'll go as far to say that if you do any work in Deep Learning (especially Convolutional Neural Networks) that you'll run into the term "Softmax": it's the final layer at the end of the network that yields your actual probability scores for each class label. To learn more about Softmax classifiers and the cross-entropy loss function, keep reading.

Artificial intelligence researchers at DeepMind have created some of the most realistic sounding human-like speech, using neural networks. Dubbed WaveNet, the AI promises significant improvements to computer-generated speech, and could eventually be used in digital personal assistants such as Siri, Cortana and Amazon's Alexa. The technology generates voices by sampling real human speech from both English and Mandarin speakers. In tests, the WaveNet generated speech was found to be more realistic than other forms of text-to-speech programs but still falling short of being truly convincing. In 500 blind tests, respondents were asked to judge sample sentences on a scale of one to five (five being most realistic).

WaveNet won't have immediate commercial applications because the system requires too much computational power: it has to sample the audio signal it is being trained on 16,000 times per second or more, DeepMind said. And then for each of those samples it has to form a prediction about what the soundwave should look like based on each of the prior samples. Even the DeepMind researchers acknowledged in their blog post that this "is a clearly challenging task."

These are the questions I often meet in my daily work with innovative startups and corporates all trying to assess the trajectories with higher potentials and market opportunities. AI is a true game changer. It has showed extraordinary advancements in the last couple of years thanks to a new technique called "deep learning" which allows machines to become superintelligent by crunching myriads of examples and data rather than being explicitly programmed. Independently of the specific deep learning technique, the idea of taking advantage from extensive amounts of computing power and giant quantities of data to mimic human brains and neuronal systems is already proving very effective to power internet search engines, block spam emails, translate web pages, recognize voice commands, etc. Moreover the recent success of AI is opening the route to applications in a number of industries: improved vision systems for example are up to boost disruption in the automotive and mobility worlds with self-driving cars, retailers logistics with smarter delivery drones, the surveillance universe and medical diagnosis scientific breakthroughs with powered image recognition. In addition dynamic-reaction systems and Automated Teller Machines can be used in massive clusters of white-collar jobs also in traditionally conservative industries such as in legal, banking, health-care, journalism, etc., sectors where human answers, expert advice, reports, editorial summaries can be substituted by Intelligent Machines.

If you've ever been lost in the maze of Youtube videos you may have stumbled on clips of computers reading news articles. You'd recognize that staccato, robotic nature of the voice. We've come a long way from "Danger! Will Robinson!," but it there is yet to be a computer that can seamlessly mimic a human voice. Now, there's a new contender, brought to you by the brilliant minds behind DeepMind.

Which authors of this paper are endorsers? Disable MathJax (What is MathJax?)

Several recent deep neural networks experiments leverage the generalist-specialist paradigm for classification. However, no formal study compared the performance of different clustering algorithms for class assignment. In this paper we perform such a study, suggest slight modifications to the clustering procedures, and propose a novel algorithm designed to optimize the performance of of the specialist-generalist classification system. Our experiments on the CIFAR-10 and CIFAR-100 datasets allow us to investigate situations for varying number of classes on similar data. We find that our \emph{greedy pairs} clustering algorithm consistently outperforms other alternatives, while the choice of the confusion matrix has little impact on the final performance.

Google's WaveNet uses neural nets to generate eerily convincing speech and music http://tcrn.ch/2ccjLjY Physicists have discovered what makes neural networks so extraordinarily powerful http://bit.ly/2cikJeB Will This "Neural Lace" Brain Implant Help Us Compete with AI? http://bit.ly/2c75cPH These nightmare videos are generated from still baby photos by a neural network http://gizmo.do/G476Lyi Mathematicians have been searching, but the answer lies in physics.

Although deep neural networks (DNNs) have achieved impressive performance in several non-trivial machine learning (ML) tasks, many challenges remain. One class of challenges has to do with understanding how and why DNNs obtain their improved performance, given that they do not exhibit properties such as convexity that are common among ML methods. A second class of challenges has to do with using this understanding to develop DNN methods that have better statistical/inferential properties and/or better algorithmic/running time properties. In this work, the team will pursue two related directions. First, to develop a model to make theoretically precise certain intuitions that might explain the performance of DNNs.