Unsupervised single-channel overlapped speech recognition is one of the hardest problems in automatic speech recognition (ASR). Permutation invariant training (PIT) is a state of the art model-based approach, which applies a single neural network to solve this single-input, multiple-output modeling problem. We propose to advance the current state of the art by imposing a modular structure on the neural network, applying a progressive pretraining regimen, and improving the objective function with transfer learning and a discriminative training criterion. The modular structure splits the problem into three sub-tasks: frame-wise interpreting, utterance-level speaker tracing, and speech recognition. The pretraining regimen uses these modules to solve progressively harder tasks. Transfer learning leverages parallel clean speech to improve the training targets for the network. Our discriminative training formulation is a modification of standard formulations, that also penalizes competing outputs of the system. Experiments are conducted on the artificial overlapped Switchboard and hub5e-swb dataset. The proposed framework achieves over 30% relative improvement of WER over both a strong jointly trained system, PIT for ASR, and a separately optimized system, PIT for speech separation with clean speech ASR model. The improvement comes from better model generalization, training efficiency and the sequence level linguistic knowledge integration.

Analysis DeepMind published a paper today describing AlphaGo Zero – a leaner and meaner version of AlphaGo, the artificially intelligent program that crushed professional Go players. Go was considered a difficult game for computers to master because, besides being complex, the number of possible moves – more than chess at 10170 – is greater than the number of atoms in the universe. But AlphaGo, the predecessor to AlphaGo Zero, crushed 18-time world champion Lee Sedol and the reigning world number one player, Ke Jie. After beating Jie earlier this year, DeepMind announced AlphaGo was retiring from future competitions. Now, an even more superior competitor is in town.

NOT so long ago, mastering the ancient Chinese game of Go was beyond the reach of artificial intelligence. But then AlphaGo, Google DeepMind's AI player, started to leave even the best human opponents in the dust. Yet even this world-beating AI needed humans to learn from. AlphaGo Zero has surpassed its predecessor's abilities, bypassing AI's traditional method of learning games, which involves watching thousands of hours of human play. Instead, it simply starts playing at random, honing its skills by repeatedly playing against itself.

Deepmind, the artificial intelligence research organization owned by Google, announced some stunning results Wednesday from research into the next generation of its AlphaGo system: the machines are getting smarter. AlphaGo Zero, the new version of the AlphaGo system that defeated the world's best Go players in competitions over the past few years, was able to teach itself how to play the ancient board game as well as its predecessors in a matter of days with no other input than the basic rules of the game, Deepmind said in a blog post Wednesday. Previous versions of AlphaGo built to compete against human masters of the game required hours and hours of training on Go gameplay, but AlphaGo Zero was able to teach itself to play using a technique called reinforcement learning. Reinforcement learning involves training a system to figure out the best reward outcome from a series of actions, unlike supervised learning, in which the system is taught which outcomes are desired and trained over and over to recognize the factors that lead to those outcomes. Deepmind set up a neural network that played games of Go against itself until it learned how to formulate a winning strategy for a game in which capturing as many stones as possible can be satisfying in early stages, but can lead to big problems as the game plays out.

High-level deep learning libraries such as TensorFlow, Keras, and Pytorch do a wonderful job in making the life of a deep learning practitioner easier by hiding many of the tedious inner-working details of neural networks. As great as this is for deep learning, it comes with the minor downside of leaving many new-comers with less foundational understanding to be learned elsewhere. Our goal here is to simply provide a 1 hidden-layer fully-connected neural network classifier written from scratch (no deep learning libraries) to help chip away that mysterious black-box feeling you might have with neural networks. The provided neural network classifies a dataset describing geometrical properties of kernels belonging to three classes of wheat (you can easily replace this with your own custom dataset). An L2-loss function is assumed, and a sigmoid transfer function is used on every node in the hidden and output layers.

Google's DeepMind says it has made another big advance in artificial intelligence by getting a machine to master the Chinese game of Go without help from human players. The AlphaGo program, devised by the tech giant's AI division, has already beaten two of the world's best players. It had started by learning from thousands of games played by humans. But the new AlphaGo Zero began with a blank Go board and no data apart from the rules, and then played itself. Within 72 hours it was good enough to beat the original program by 100 games to zero. DeepMind's chief executive, Demis Hassabis, said the system could now have more general applications in scientific research.

DARPA (Defense Advanced Research Projects Agency), is a division of the American Defense Department that investigates new technologies. It has for some time regarded the current generation of AI technologies as important in the future. It has been in the forefront of AI research in image recognition, speech recognition and generation, robotics, autonomous vehicles, medical diagnostic systems, and more. However, DARPA is well aware that despite the high level of problem-solving capabilities of AI programs – they lack explainability. AI deep learning algorithms use complex mathematics that is very difficult for human users to understand or comprehend.

Our program, AlphaGo Zero, differs from AlphaGo Fan and AlphaGo Lee 12 in several important aspects. First and foremost, it is trained solely by self-play reinforcement learning, starting from random play, without any supervision or use of human data. Second, it only uses the black and white stones from the board as input features. Third, it uses a single neural network, rather than separate policy and value networks. Finally, it uses a simpler tree search that relies upon this single neural network to evaluate positions and sample moves, without performing any MonteCarlo rollouts.

In the 1970 sci-fi thriller Colossus: The Forbin Project, a computer designed to control the United States' nuclear weapons is switched on, and immediately discovers the existence of a Soviet counterpart. The two machines have become one, and it has mankind by the throat. Development work takes a lot longer than that. Today DeepMind, a London-based subsidiary of Google, announced that it has developed a machine that plays the ancient Chinese game of Go much better than its predecessor, AlphaGo, which last year beat Lee Sedol, a world-class player, in Seoul. The earlier program was trained for months on a massive database of master games and got plenty of pointers--training wheels, as it were--from its human creators.

Artificial intelligence is where the competition is in IT, with Microsoft and Google both parading powerful, always-available AI tools for the enterprise at their respective developer conferences, Build and I/O, in May. It's not just about work: AI software can now play chess, go, and some retro video games better than any human -- and even drive a car better than many of us. These superhuman performances, albeit in narrow fields, are all possible thanks to the application of decades of AI research -- research that is increasingly, as at Build and I/O, making it out of the lab and into the real world. Alexa and Samsung Electronics' Bixby may offer less-than-superhuman performance, but they also require vastly less power than a supercomputer to run. Businesses can dabble on the edges of these, for example developing Alexa "skills" that allow Amazon Echo owners to interact with a company without having to dial its call center, or jump right in, using the various cloud-based speech recognition and text-to-speech "-as-a-service" offerings to develop full-fledged automated call centers of their own. Some of the earliest work on AI sought to explicitly model human knowledge of the world in a form that computers could process and reason from, if not actually understand.