Genre
Invariant Representations for Noisy Speech Recognition
Serdyuk, Dmitriy, Audhkhasi, Kartik, Brakel, Philémon, Ramabhadran, Bhuvana, Thomas, Samuel, Bengio, Yoshua
Modern automatic speech recognition (ASR) systems need to be robust under acoustic variability arising from environmental, speaker, channel, and recording conditions. Ensuring such robustness to variability is a challenge in modern day neural network-based ASR systems, especially when all types of variability are not seen during training. We attempt to address this problem by encouraging the neural network acoustic model to learn invariant feature representations. We use ideas from recent research on image generation using Generative Adversarial Networks and domain adaptation ideas extending adversarial gradient-based training. A recent work from Ganin et al. proposes to use adversarial training for image domain adaptation by using an intermediate representation from the main target classification network to deteriorate the domain classifier performance through a separate neural network. Our work focuses on investigating neural architectures which produce representations invariant to noise conditions for ASR. We evaluate the proposed architecture on the Aurora-4 task, a popular benchmark for noise robust ASR. We show that our method generalizes better than the standard multi-condition training especially when only a few noise categories are seen during training.
Accurate De Novo Prediction of Protein Contact Map by Ultra-Deep Learning Model
Wang, Sheng, Sun, Siqi, Li, Zhen, Zhang, Renyu, Xu, Jinbo
Recently exciting progress has been made on protein contact prediction, but the predicted contacts for proteins without many sequence homologs is still of low quality and not very useful for de novo structure prediction. This paper presents a new deep learning method that predicts contacts by integrating both evolutionary coupling (EC) and sequence conservation information through an ultra-deep neural network formed by two deep residual networks. This deep neural network allows us to model very complex sequence-contact relationship as well as long-range inter-contact correlation. Our method greatly outperforms existing contact prediction methods and leads to much more accurate contact-assisted protein folding. Tested on three datasets of 579 proteins, the average top L long-range prediction accuracy obtained our method, the representative EC method CCMpred and the CASP11 winner MetaPSICOV is 0.47, 0.21 and 0.30, respectively; the average top L/10 long-range accuracy of our method, CCMpred and MetaPSICOV is 0.77, 0.47 and 0.59, respectively. Ab initio folding using our predicted contacts as restraints can yield correct folds (i.e., TMscore>0.6) for 203 test proteins, while that using MetaPSICOV- and CCMpred-predicted contacts can do so for only 79 and 62 proteins, respectively. Further, our contact-assisted models have much better quality than template-based models. Using our predicted contacts as restraints, we can (ab initio) fold 208 of the 398 membrane proteins with TMscore>0.5. By contrast, when the training proteins of our method are used as templates, homology modeling can only do so for 10 of them. One interesting finding is that even if we do not train our prediction models with any membrane proteins, our method works very well on membrane protein prediction. Finally, in recent blind CAMEO benchmark our method successfully folded 5 test proteins with a novel fold.
Gartner's Top 10 Strategic Technology Trends For 2017
Nintendo Reports Second Quarter Losses But 3DS Sales Are Up Thanks To'Pokmon GO' Increasingly, the world is becoming an intelligent, digitally enabled mesh of people, things and services. Technology will be embedded in everything in the digital business of the future, and ordinary people will experience a digitally-enabled world where the lines between what is real and what is digital blur. Rich digital services will be delivered to everything, and intelligence will be embedded in everything behind the scenes. We call this mesh of people, devices, content and services the intelligent digital mesh, and this forms the basis for our Top 10 Strategic Technology Trends for 2017. Artificial Intelligence (AI) and machine learning have reached a critical tipping point and will increasingly augment and extend virtually every technology enabled service, thing or application.
An Interactive Tutorial on Numerical Optimization
Numerical Optimization is one of the central techniques in Machine Learning. For many problems it is hard to figure out the best solution directly, but it is relatively easy to set up a loss function that measures how good a solution is - and then minimize the parameters of that function to find the solution. I ended up writing a bunch of numerical optimization routines back when I was first trying to learn javascript. Since I had all this code lying around anyway, I thought that it might be fun to provide some interactive visualizations of how these algorithms work. The cool thing about this post is that the code is all running in the browser, meaning you can interactively set hyper-parameters for each algorithm, change the initial location, and change what function is being called to get a better sense of how these algorithms work.
Three Original Math and Proba Challenges, with Tutorial
Here I offer a few off-the-beaten-path interesting problems that you won't find in textbooks, data science camps, or in college classes. These problems range from applied maths, to statistics and computer science, and are aimed at getting the novice interested in a few core subjects that most data scientists master. The problems are described in simple English and don't require math / stats / probability knowledge beyond high school level. My goal is to attract people interested in data science, but who are somewhat concerned by the depth and volume of (in my opinion) unnecessary mathematics included in many curricula. I believe that successful data science can be engineered and deployed by scientists coming from other disciplines, who do not necessarily have a deep analytical background yet are familiar with data.
Artificial intelligence has a lot to learn from babies
This article originally appeared on the International Business Times. Machines are capable of understanding speech, recognizing faces and driving cars safely, making recent technological advancements seem impressively powerful. But if the field of artificial intelligence is going to make the transformative leap into building human-like machines, it'll first have to master the way babies learn. "Relatively recently in AI there's been a shift from thinking about designing systems that can do the sort of things that adults can do, to realizing if you want to have systems that are as flexible and powerful and do the kinds of things that adults do, you need to have systems that can learn the way babies and children do," developmental psychologist Alison Gopnik, a researcher at the University of California at Berkeley, told International Business Times. "If you compare what computers can do now to what they could do 10 years ago, they've certainly made a lot of progress, but if you compare them to what a 4-year-old can do, there's still a pretty enormous gap."
Carnegie Mellon And Yale Robot Experts Release Robotics Roadmap Report
While robots have the potential to be very intelligent, if there's one thing that books, movies, and even our own experiences have shown, it's that they also can be remarkably dumb. So one sure-fire way for the U.S. to continue leading the robotics world is by investing in education. Not only are these machines getting smarter every day, but so too are other countries, training the kind of workers required to operate robots that can coat cars on assembly lines with paint and produce sneakers faster than ever. That's one takeaway from a new report released Monday by a group of 120 robotics experts. Intended to brief the U.S. government on the state of robotics so that the government can better plan for the future, the Roadmap to Robotics report is sponsored by the National Science Foundation (as well as a few universities) and written by experts from the private sector as well as academic institutions like like Carnegie Mellon, Georgia Institute of Technology, University of California, Berkeley, and Yale.
The invention of Artificial Intelligence and what it means for the world of work?
The story of Artificial Intelligence is as old as Greek antiquity (Greek myths incorporated the idea of intelligent robots). It all began with myths, stories and rumors of artificial beings gifted with intelligence by master craftsmen. As Pamela McCorduck puts it: "Artificial Intelligence began with an ancient wish to forge the Gods." McCorduck was popular for penning books concerning the history and philosophical significance of Artificial Intelligence. But, when were the seeds of modern Artificial Intelligence planted? In the 1940s, a programmable digital computer was invented – a machine based on the abstract essence of mathematical reasoning.
Google's AI just created its own universal 'language'
Google has previously taught its artificial intelligence to play games, and it's even capable of creating its own encryption. Now, its language translation tool has used machine learning to create a'language' all of its own. In September, the search giant turned on its Google Neural Machine Translation (GNMT) system to help it automatically improve how it translates languages. The machine learning system analyses and makes sense of languages by looking at entire sentences – rather than individual phrases or words. Following several months of testing, the researchers behind the AI have seen it be able to blindly translate languages even if it's never studied one of the languages involved in the translation.
100 Blogs on Analytics, Big Data, Data Science, and Machine Learning
We've added some blogs that were missing in the original list, and eliminated some that aren't worth mentioning, hoping to make this list less biased. AnalyticBridge, about advanced analytics, books, salary surveys, training, challenges. Anil Batra's Web Analysis (Analytics), Online Advertising and Behavioral Targeting blog BigDataNews General articles about big data, as well as news (selected press releases) Business. CoolData By Kevin MacDonell on Analytics, predictive modeling and related cool data stuff for fund-raising in higher education. Cloud of data blog By Paul Miller, aims to help clients understand the implications of taking data and more to the Cloud.