"Many researchers … speculate that the information-processing abilities of biological neural systems must follow from highly parallel processes operating on representations that are distributed over many neurons. [Artificial neural networks] capture this kind of highly parallel computation based on distributed representations"
– from Machine Learning (Section 4.1.1; page 82) by Tom M. Mitchell, McGraw Hill Companies, Inc. (1997).
Is it possible to understand the brain? Science is still far from answering this question. New artificial neural networks developed by the Max Planck Institute of Neurobiology and Google AI can now even recognize and classify nerve cells independently based on their appearance. The human brain consists of about 86 billion nerve cells and about as many glial cells. In addition, there are about 100 trillion connections between the nerve cells alone.
As deep learning has become ubiquitous, evaluations of its accuracy typically compare its performance against an idealized baseline of flawless human results that bear no resemblance to the actual human workflow those algorithms are being designed to replace. For example, the accuracy of real-time algorithmic speech recognition is frequently compared against human captioning produced in offline multi-coder reconciled environments and subjected to multiple reviews to generate flawless content that looks absolutely nothing like actual real-time human transcription. If we really wish to understand the usability of AI today we should be comparing it against the human workflows it is designed to replace, not an impossible vision of nonexistent human perfection. While the press is filled with the latest superhuman exploits of bleeding-edge research AI systems besting humans at yet another task, the reality of production AI systems is far more mundane. Most commercial applications of deep learning can achieve higher accuracy than their human counterparts at some tasks and worse performance on others.
When asked why he robbed banks, Willie Sutton famously replied, "Because that's where the money is". And so much of artificial antelligence evolved in the United States – because that's where the computers were. However with Europe's strong educational institutions, the path to advanced AI technologies has been cleared by European computer scientists, neuroscientists, and engineers – many of whom were later poached by US universities and companies. From backpropagation to Google Translate, deep learning, and the development of more advanced GPUs permitting faster processing and rapid developments in AI over the past decade, some of the greatest contributions to AI have come from European minds. Modern AI can be traced back to the work of the English mathematician Alan Turing, who in early 1940 designed the bombe – an electromechanical precursor to the modern computer (itself based on previous work by Polish scientists) that broke the German military codes in World War II.
It seems like using these pre-trained models have become a new standard for industry best practices. After all, why wouldn't you take advantage of a model that's been trained on more data and compute than you could ever muster by yourself? Advances within the NLP space have also encouraged the use of pre-trained language models like GPT and GPT-2, AllenNLP's ELMo, Google's BERT, and Sebastian Ruder and Jeremy Howard's ULMFiT (for an excellent over of these models, see this TOPBOTs post). One common technique for leveraging pretrained models is feature extraction, where you're retrieving intermediate representations produced by the pretrained model and using those representations as inputs for a new model. These final fully-connected layers are generally assumed to capture information that is relevant for solving a new task.
Danelle is CMO at Blue Hexagon. She has more than 15 years of experience bringing new technologies to market. Prior to Blue Hexagon, Danelle was VP Marketing at SafeBreach where she built the marketing team and defined the Breach and Attack Simulation category. Previously, she led strategy and marketing at Adallom, a cloud security company acquired by Microsoft. She was also Director, Security Solutions at Palo Alto Networks, driving growth in critical IT initiatives like virtualization, network segmentation and mobility.
Why are deep neural networks hard to train? Appendix: Is there a simple algorithm for intelligence? If you benefit from the book, please make a small donation. I suggest $5, but you can choose the amount. Thanks to all the supporters who made the book possible, with especial thanks to Pavel Dudrenov. In the last chapter we learned that deep neural networks are often much harder to train than shallow neural networks. That's unfortunate, since we have good reason to believe that if we could train deep nets they'd be much more powerful than shallow nets. But while the news from the last chapter is discouraging, we won't let it stop us. In this chapter, we'll develop techniques which can be used to train deep networks, and apply them in practice. We'll also look at the broader picture, briefly reviewing recent progress on using deep nets for image recognition, speech recognition, and other applications. And we'll take a brief, speculative look at what the future may hold for neural nets, ...
Deep learning is revolutionizing medicine. Algorithms are increasingly doing everything from triaging medical imagery to predicting treatment outcomes. Yet as hospitals undergo the same AI revolution affecting other fields, the dangers of AI bias and errors and the life-or-death consequences of medicine lends unique risk to these experiments, suggesting caution. One of the fastest-growing uses of AI in medicine today is the analysis of medical imagery. Human analysis of imagery is slow, difficult to scale and error-prone.
Tensorflow inspires developers to experiment with their exciting AI ideas in almost any domain that comes to mind. There are three well known factors in the ML community that make up a good Deep Neural Network model do magical things. My area of interest is Real Time Communication. Coming up with practical ML use cases that may add value to RTC applications is the easy part. I wrote about a few of these recently.
As technology goes more and more towards Artificial Intelligence branches such as Machine Learning and Deep Learning technologies, data and information are getting more and more endangered by Fake News and tampered materials. Deep Fakes seems to be the most "promising" and dangerous example of this kind. Shortly, it allows creating a tampered video content by replace its behavior. Say, like replacing an actor's face with someone's else, as it happened to Gal Gadot (read more). Turns out there are out thousands of "faked" videos out in the network.
This is the first part of a three-part blog series. You can read the second part here and the third part here. The Internet of Things provides us with lots of sensor data. However, the data by itself does not provide value unless we can turn it into actionable, contextualized information. Big data and data visualization techniques allow us to gain new insights through batch-processing and off-line analysis.