In this blog, I will try to explain what is deep learning and how it relates to machine learning. Even though there's a lot of knowledge about the subject online, I could not point to one source that was both exhaustive and concise. Before I begin explaining what exactly is deep learning, I'll first explain a little about artificial intelligence, machine learning, neural networks and the relationship between these topics Artificial Intelligence (AI) is the scientific branch that emphasizes the development of creating machines that can operate similarly to humans. Type 1, the simplest type, it can decide without memory and the use of prior experience. This type of intelligence is reactive, meaning it can only react to a current situation.

Think it's hard keeping up with your Twitter feed? Imagine keeping track of all of Twitter. "Every tweet is comprised of over 100 data points," says Matt Singer, a senior staff hardware engineer responsible for server architecture at Twitter. Data from every retweet, "unfollow", link-click and other actions feeds analytic and deep learning systems serving operational, advertising. How does an organization handle such hyper-scale demands?

This post originally published on Medium. It is republished here with permission. I've been around computing since my older brother got a Commodore 64 for Christmas in 1983. I took my first "business machines" class in high school in 1991, attended my first computer science class in 1994 (learning Pascal), and moved to Silicon Valley in 1997 after Cisco converted my internship into a permanent position. I worked in Cisco's IT department for several years before moving to their engineering group, where I designed networking protocols. I went to grad school at MIT in 2004, where I met the founders of several companies in Y Combinator's first couple of batches and worked on Hubspot before it was Hubspot. After writing several books for O'Reilly and attending the first O'Reilly Web 2.0 and MIT Sloan Sports Analytics conferences, I started a "Web 2.0 for Sports" company called StatSheet.com in 2007, which, in 2010, pivoted into the first Natural Language Generation (NLG) company called Automated Insights. I recently stepped back at Automated Insights to become a Ph.D. student at UNC studying artificial intelligence.

Sign in to report inappropriate content. Now that we've learned about the basic feed forward, fully connected, neural network, it's time to cover a new one: the convolutional neural network, often referred to as a convnet or cnn. Convolutional neural networks got their start by working with imagery.

Now that we've learned about the basic feed forward, fully connected, neural network, it's time to cover a new one: the convolutional neural network, often referred to as a convnet or cnn. Convolutional neural networks got their start by working with imagery. The idea of doing image analysis is to recognize things like objects, such as humans, or cars. In order to recognize a car or a human, it's fairly challenging to do so if you're thinking of things 1 pixel at a time. Instead, a convolutional neural network aims to use a sliding window (a kernel) that takes into account a group of pixels, to hopefully recognize small features like "edges" or "curves" and then another layer might take combinations of edges or curves to detect shapes like squares or circles, or other complex types of shapes...and so on.

It has, and it will generate even more. But there are lots of problems that narrow AI just doesn't seem very capable of. Things like conversational natural-language understanding and general assistance in the virtual world, or things like Rosie the robot that might be able to help you tidy your home or cook you dinner. Those are just way outside of the scope of what we can do with narrow AI. It's also an interesting empirical question about whether narrow AI can get us to safe driverless cars. The reality so far is that narrow AI has a lot of problems with outlier cases, even for driving, which is a fairly constrained problem.

THE ECONOMIST'S Open Future essay competition winner was announced in September, beating nearly 2,400 entries from over 110 countries. But how might artificial intelligence tackle the question? Specifically, we fed the essay question and the 58-word description through a natural-language processing algorithm called GPT-2, released publicly in February by OpenAI, a group working on AI research and ethics, based in San Francisco. The result was six roughly 400-word texts. We took the larger parts of three of them and placed them one after another with no other editing.

False Data Injection Attacks in Internet of Things and Deep Learning enabled Predictive Analytics Gautam Raj Mode, Prasad Calyam, Khaza Anuarul Hoque Department of Electrical Engineering & Computer Science University of Missouri, Columbia, MO, USA gmwyc@mail.missouri.edu, Abstract --Industry 4.0 is the latest industrial revolution primarily merging automation with advanced manufacturing to reduce direct human effort and resources. Predictive maintenance (PdM) is an industry 4.0 solution, which facilitates predicting faults in a component or a system powered by state-of-the-art machine learning (ML) algorithms (especially deep learning algorithms) and the Internet-of-Things (IoT) sensors. However, IoT sensors and deep learning (DL) algorithms, both are known for their vulnerabilities to cyber-attacks. In the context of PdM systems, such attacks can have catastrophic consequences as they are hard to detect due to the nature of the attack. T o date, the majority of the published literature focuses on the accuracy of the IoT and DL enabled PdM systems and often ignores the effect of such attacks. In this paper, we demonstrate the effect of IoT sensor attacks (in the form of false data injection attack) on a PdM system. At first, we use three state-of-the-art DL algorithms, specifically, Long Short-T erm Memory (LSTM), Gated Recurrent Unit (GRU), and Convolutional Neural Network (CNN) for predicting the Remaining Useful Life (RUL) of a turbofan engine using NASA's C-MAPSS dataset. Our obtained results show that the GRU-based PdM model outperforms some of the recent literature on RUL prediction using the C-MAPSS dataset. Afterward, we model and apply two different types of false data injection attacks (FDIA), specifically, continuous and interim FDIAs on turbofan engine sensor data and evaluate their impact on CNN, LSTM, and GRU-based PdM systems. Our results demonstrate that attacks on even a small number of IoT sensors can strongly defect the RUL prediction in all cases. However, the GRU-based PdM model performs better in terms of accuracy and FDIA resiliency. Lastly, we perform a study on the GRU-based PdM model using four different GRU networks with different sequence lengths.

We propose a novel defense against all existing gradient based adversarial attacks on deep neural networks for image classification problems. Our defense is based on a combination of deep neural networks and simple image transformations. While straight forward in implementation, this defense yields a unique security property which we term buffer zones. In this paper, we formalize the concept of buffer zones. We argue that our defense based on buffer zones is secure against state-of-the-art black box attacks. We are able to achieve this security even when the adversary has access to the {\em entire} original training data set and unlimited query access to the defense. We verify our security claims through experimentation using FashionMNIST, CIFAR-10 and CIFAR-100. We demonstrate $<10\%$ attack success rate -- significantly lower than what other well-known defenses offer -- at only a price of a 15-20\% drop in clean accuracy. By using a new intuitive metric we explain why this trade-off offers a significant improvement over prior work.

Recent advances in pre-training huge models on large amounts of text through self supervision have obtained state-of-the-art results in various natural language processing tasks. However, these huge and expensive models are difficult to use in practise for downstream tasks. Some recent efforts use knowledge distillation to compress these models. However, we see a gap between the performance of the smaller student models as compared to that of the large teacher. In this work, we leverage large amounts of in-domain unlabeled transfer data in addition to a limited amount of labeled training instances to bridge this gap. We show that simple RNN based student models even with hard distillation can perform at par with the huge teachers given the transfer set. The student performance can be further improved with soft distillation and leveraging teacher intermediate representations. We show that our student models can compress the huge teacher by up to 26x while still matching or even marginally exceeding the teacher performance in low-resource settings with small amount of labeled data.