"The field of Machine Learning seeks to answer these questions: How can we build computer systems that automatically improve with experience, and what are the fundamental laws that govern all learning processes?"
– from The Discipline of Machine Learning by Tom Mitchell. CMU-ML-06-108, 2006.
In this tutorial, you will learn the three primary reasons your validation loss may be lower than your training loss when training your own custom deep neural networks. I first became interested in studying machine learning and neural networks in late high school. Back then there weren't many accessible machine learning libraries -- and there certainly was no scikit-learn. Every school day at 2:35 PM I would leave high school, hop on the bus home, and within 15 minutes I would be in front of my laptop, studying machine learning, and attempting to implement various algorithms by hand. I rarely stopped for a break, more than occasionally skipping dinner just so I could keep working and studying late into the night.
"I bought the first version of this book, and now also the second. The new version is very comprehensive. If you are using Python - it's almost a reference. I also like the emphasis on neural networks (and TensorFlow) - which (in my view) is where the Python community is heading. I am also planning to use this book in my teaching at Oxford University. The data pre-processing sections are also good. I found the sequence flow slightly unusual - but for an expert level audience, it's not a major issue."
What a time this is to be working in the machine learning field! The last few years have been a dream run for anyone associated with machine learning as there have been a slew of developments and breakthroughs at an unprecedented pace. There's just one thing to keep in mind here – these breakthroughs did not happen overnight. It took years and in some cases, decades, of hard work and persistence. We are used to working with established machine learning algorithms like neural networks and random forest (and so on). We tend to lose sight of the effort it took to make these algorithms mainstream. To actually create them from scratch. The people who lay the groundwork for us – those are the true heroes of machine learning.
We need to get a taste of machine learning before understanding it fully. This segment is made up of three parts. These are not the exact courses I took to learn Python and getting an intro to data science. But they are quite similar and they serve the purpose. This course is about learning to use Python and creating things on your own.
I am the co-founder and CEO of Neurala Inc., a Boston-based company building Artificial Intelligence emulating brain function in software. Neurala's deep learning tech makes robots, drones, cars, consumer electronics, toys and smart devices more useful, engaging and autonomous. Neurala stems out of 10 years of research at Boston University Neuromorphics Lab, where as AI Professor I have pioneered the research and fielding of brain-inspired (also called Deep Learning, or Artificial Neural Networks) algorithms that allow robots and drones to perceive, navigate, interact and learn real-time in complex environments. Over my academic and industrial career, I have lectured and spoken at dozens of events and venues, including TEDx, keynote at Mobile World Congress Drone Summit, NASA, the Pentagon, GTC, InterDrone, Los Alamo National Lab, GE, Air Force Research Labs, HP, iRobot, Samsung, LG, Qualcomm, Huawei, Ericsson, BAE Systems, AI World, Mitsubishi, ABB and Accenture, among many others. My work has been featured in TIME, IEEE Spectrum, Fortune, CNBC, The Boston Globe, Xconomy, The Chicago Tribune, TechCrunch, VentureBeat, Nasdaq, Associated Press and many other media.
Keras is a simple-to-use but powerful deep learning library for Python. In this post, we'll build a simple Convolutional Neural Network (CNN) and train it to solve a real problem with Keras. This post is intended for complete beginners to Keras but does assume a basic background knowledge of CNNs. My introduction to Convolutional Neural Networks covers everything you need to know (and more) for this post - read that first if necessary. The full source code is at the end.
Recently on our podcast we talked about a conversation Tim Peter had with a colleague. The discussion revolved around AI. Tim was asked an intriguing question. All this AI stuff, when we talk about marketing and all that, that's all just hype, right?" The suggestion was that AI is just not a real thing that matters yet. But he's not sure it matters today. He couldn't be more wrong. If you don't have a sense of how big AI is, I suggest you go drive a car. Not just any car, but a brand new car–they are not the same machine as 10 years ago. You don't have to choose a Tesla; Subarus are a great example of how AI is making driving better, safer. But set the adaptive cruise control. There is a model in there built using machine learning that uses data about the environment and the capabilities of the vehicle to make driving easier and safer. The model is continuously making predictions. It asks, "Am I going to hit something?
One of the big benefits of an ML-based system is that it becomes more intelligent as it "learns" patterns of use, actions, and storage. The objective of data scientists who build ML-based search and analysis systems is, of course, to get as close as possible to zero errors and produce answers in as short a time as possible. Of course, nothing is perfect, and each type of machine learning system has its foibles – biases, bad data inputs, clustering issues, and others – but good segmentation, proper training sets, can improve the situation. But ensuring that staff remains alert when the machines begin to augment their work is likely to become more of a challenge, if our experience with technology in the past is any indication.
Artificial Intelligence (AI) is the mantra of the current era. The phrase is intoned by technologists, academicians, journalists, and venture capitalists alike. As with many phrases that cross over from technical academic fields into general circulation, there is significant misunderstanding accompanying use of the phrase. However, this is not the classical case of the public not understanding the scientists--here the scientists are often as befuddled as the public. The idea that our era is somehow seeing the emergence of an intelligence in silicon that rivals our own entertains all of us, enthralling us and frightening us in equal measure. There is a different narrative that one can tell about the current era.