Deep Learning
TensorFlow in a Nutshell -- Part One: Basics
TensorFlow is a framework created by Google for creating Deep Learning models. Deep Learning is a category of machine learning models that use multi-layer neural networks. The idea of deep learning has been around since 1943 when neurophysiologist Warren McCulloch and mathematician Walter Pitts wrote a paper on how neurons might work and they modeled a simple neural network using electrical circuits. Many, many developments have occurred since then. These highly accurate mathematical models are extremely computationally expensive.
Accelerated Computing and Deep Learning
Intelligent machines powered by AI computers that can learn, reason and interact with people are no longer science fiction. Today, a self-driving car powered by AI can meander through a country road at night and find its way. An AI-powered robot can learn motor skills through trial and error. This is truly an extraordinary time. In my three decades in the computer industry, none has held more potential, or been more fun. The era of AI has begun.
AI vs Deep Learning vs Machine Learning2
Which of the following are substantially the same things? For as precise a profession as we data scientists purport to be we are sometimes way too casual with our language. Read several articles about AI, Deep Learning, and Machine learning and you will come away confused whether these are all the same or all different. Imagine how confused non-data scientists must be. The truth is that each of these terms has some overlap in a Venn diagram but none of these is a perfect subset of the other and none completely explains the others.
WTF is machine learning?
While the number of headlines about machine learning might lead one to think that we just discovered something profoundly new, the reality is that the technology is nearly as old as computing. It's no coincidence that Alan Turing, one of the most influential computer scientists of all time, started his 1950 treatise on computing with the question "Can machines think?" From our science fiction to our research labs, we have long questioned whether the creation of artificial versions of ourselves will somehow help us uncover the origin of our own consciousness, and more broadly, our role on earth. Unfortunately, the learning curve on AI is really damn steep. By tracing a bit of history, we should hopefully be able to get to the bottom of wtf machine learning really is.
Deep Learning Drives General Artificial Intelligence
Mountain View, California-based Drive.ai is a startup created by former lab mates from Stanford University's Artificial Intelligence Lab. Originally founded in 2015 by Carol Reiley and Fred Rosenzweig, Drive.ai raised $12 million in Series A funding earlier this year to develop deep learning algorithms to control the operation of autonomous vehicles. Building on experience gained from the DARPA Grand Challenge, Google and other self-driving pioneers programmed the first self-driving car to rely primarily on light detection and ranging (LIDAR), which is a remote sensing method that uses pulses of laser light to measure distances, and detailed mapping. Although this has worked pretty well, the current technology is expensive. Making autonomous vehicles easier to manufacture with less expensive parts will make them more affordable.
Nightmare Machine taps AI to make ordinary photos horrifying
As a San Francisco resident, I've often been awed and inspired by the sight of the Golden Gate Bridge. A team from MIT's Media Lab ran a photo of the landmark through its "Nightmare Machine" and now it looks like a moving, tentacled monster that will grab and crush any car that dares to cross it. The Nightmare Machine uses deep-learning algorithms (and possibly evil spirits) to turn ordinary images of people and places into scary ones. To help the AI learn maximum spookiness, the public is invited to rate the faces as "scary" or "not scary." The Halloween-perfect project comes from MIT Media Lab's Scalable Cooperation group, which studies how technology is reshaping the nature of human cooperation.
Microsoft makes its deep learning tools available to all
The same internal, deep learning tools that Microsoft engineers used to build its human-like speech recognition engine, as well as consumer products like Skype Translator and Cortana, are now available for public use. Redmond announced today that it is open-sourcing the Cognitive Toolkit that has led to many key developments coming out of its dedicated AI division. In other words: anyone can now train their own artificial intelligence. Formerly known as the CNTK, Microsoft says the beta version of the Cognitive Toolkit is not only faster than previous incarnations, but it is also beats out competing deep learning toolkits – especially when crunching large datasets across multiple machines. On a more practical level for startups and hobbyists, Microsoft says the platform is flexible enough to run on a solo laptop -- just in case you don't have a server farm loaded with NVIDIA GPUs at your disposal.
Here's why artificial intelligence isn't out to get us
AI has a long way to go before people can or should worry about turning the world over to machines. Elon Musk's new plan to go all-in on self-driving vehicles puts a lot of faith in the artificial intelligence needed to ensure his Teslas can read and react to different driving situations in real time. AI is doing some impressive things--last week, for example, makers of the AlphaGo computer program reported that their software has learned to navigate the intricate London subway system like a native. Even the White House has jumped on the bandwagon, releasing a report days ago to help prepare the U.S. for a future when machines can think like humans. But AI has a long way to go before people can or should worry about turning the world over to machines, says Oren Etzioni, a computer scientist who has spent the past few decades studying and trying to solve fundamental problems in AI.
Spooky algorithm transforms famous sights into horror scenes
The AI'nightmare machine': Spooky Google algorithm transforms famous sights into horror scenes The DeepDream algorithm transfers a photograph of the Eiffel Tower in Paris to a horror scene, in a style called'Fright Night', according to the website. 'We use state-of-the-art deep learning algorithms to learn how haunted houses, or toxic cities look like,' the researchers said Interested viewers can help MIT find the essence of horror on the website, or look at more of the pictures the Nightmare Machine has generated on Instagram. A normal photograph of St Basil's Cathedral in Moscow is pictured left. The Nightmare Machine team is making photographs of famous landmarks appear scary. In creating a network that works against itself, researchers believe it will eventually learn to be more precise in its output.
Residual Networks Behave Like Ensembles of Relatively Shallow Networks
Veit, Andreas, Wilber, Michael, Belongie, Serge
In this work we propose a novel interpretation of residual networks showing that they can be seen as a collection of many paths of differing length. Moreover, residual networks seem to enable very deep networks by leveraging only the short paths during training. To support this observation, we rewrite residual networks as an explicit collection of paths. Unlike traditional models, paths through residual networks vary in length. Further, a lesion study reveals that these paths show ensemble-like behavior in the sense that they do not strongly depend on each other. Finally, and most surprising, most paths are shorter than one might expect, and only the short paths are needed during training, as longer paths do not contribute any gradient. For example, most of the gradient in a residual network with 110 layers comes from paths that are only 10-34 layers deep. Our results reveal one of the key characteristics that seem to enable the training of very deep networks: Residual networks avoid the vanishing gradient problem by introducing short paths which can carry gradient throughout the extent of very deep networks.