Deep Learning
Deep learning in satellite imagery
In this article, I hope to inspire you to start exploring satellite imagery datasets. Recently, this technology has gained huge momentum, and we are finding that new possibilities arise when we use satellite image analysis. Satellite data changes the game because it allows us to gather new information that is not readily available to businesses. Satellite images allow you to view Earth from a broader perspective. You can point to any location on Earth and get the latest satellite images of that area. Also, this information is easy to access.
Waymo tests AI driving system that learns from labeled data
Alphabet's self-driving spinoff Waymo achieved some noteworthy milestones this year, in August surpassing 10 million real-world miles with its driverless cars and last week launching Waymo One, a commercial driverless taxi service. But its researchers have their eyes fixed on the future. In a blog post published today on Medium, researchers Mayank Bansal and Abhijit Ogale detailed an approach to AI driver training that taps labeled data -- that is to say, Waymo's millions of annotated miles from expert driving demonstrations -- in a supervised manner. "In recent years, the supervised training of deep neural networks using large amounts of labeled data has rapidly improved the state-of-the-art in many fields, particularly in the area of object perception and prediction, and these technologies are used extensively at Waymo," the researchers wrote. "Following the success of neural networks for perception, we naturally asked ourselves the question: โฆ can we train a skilled driver using a purely supervised deep learning approach?"
Why artificial intelligence is likely to take more lives
Artificial neurons for deeply intelligent machines โ this is the new artificial intelligence (AI) revolution, led by Geoffrey Hinton and his team since 2012. That year, Hinton, an expert in cognitive science at the University of Toronto and a researcher at Google Brain, demonstrated the striking effectiveness of a deep neural network (DNN) in an image-categorisation task. In the wake of these remarkable results, universities โ and international corporations โ invested massively in the promising and fascinating field of AI. Yet despite the impressive performance of DNNs in a variety of fields (visual and vocal recognition, translation, medical imagery, etc.), questions remain regarding the limits of deep learning for other uses, such as antonymous vehicles. To understand the limits of AI in its current state, we need to understand where DNNs come from and, above all, which areas of the human brain they are modelled on โ little is known about this in industrial engineering, and even in some research centres.
Feedback alignment in deep convolutional networks
Moskovitz, Theodore H., Litwin-Kumar, Ashok, Abbott, L. F.
Ongoing studies have identified similarities between neural representations in biological networks and in deep artificial neural networks. This has led to renewed interest in developing analogies between the backpropagation learning algorithm used to train artificial networks and the synaptic plasticity rules operative in the brain. These efforts are challenged by biologically implausible features of backpropagation, one of which is a reliance on symmetric forward and backward synaptic weights. A number of methods have been proposed that do not rely on weight symmetry but, thus far, these have failed to scale to deep convolutional networks and complex data. We identify principal obstacles to the scalability of such algorithms and introduce several techniques to mitigate them. We demonstrate that a modification of the feedback alignment method that enforces a weaker form of weight symmetry, one that requires agreement of weight sign but not magnitude, can achieve performance competitive with backpropagation. Our results complement those of Bartunov et al. (2018) and Xiao et al. (2018b) and suggest that mechanisms that promote alignment of feedforward and feedback weights are critical for learning in deep networks.
A Style-Based Generator Architecture for Generative Adversarial Networks
Karras, Tero, Laine, Samuli, Aila, Timo
We propose an alternative generator architecture for generative adversarial networks, borrowing from style transfer literature. The new architecture leads to an automatically learned, unsupervised separation of high-level attributes (e.g., pose and identity when trained on human faces) and stochastic variation in the generated images (e.g., freckles, hair), and it enables intuitive, scale-specific control of the synthesis. The new generator improves the state-of-the-art in terms of traditional distribution quality metrics, leads to demonstrably better interpolation properties, and also better disentangles the latent factors of variation. To quantify interpolation quality and disentanglement, we propose two new, automated methods that are applicable to any generator architecture. Finally, we introduce a new, highly varied and high-quality dataset of human faces.
Detecting weak and strong Islamophobic hate speech on social media
Islamophobic hate speech on social media inflicts considerable harm on both targeted individuals and wider society, and also risks reputational damage for the host platforms. Accordingly, there is a pressing need for robust tools to detect and classify Islamophobic hate speech at scale. Previous research has largely approached the detection of Islamophobic hate speech on social media as a binary task. However, the varied nature of Islamophobia means that this is often inappropriate for both theoretically-informed social science and effectively monitoring social media. Drawing on in-depth conceptual work we build a multi-class classifier which distinguishes between non-Islamophobic, weak Islamophobic and strong Islamophobic content. Accuracy is 77.6% and balanced accuracy is 83%. We apply the classifier to a dataset of 109,488 tweets produced by far right Twitter accounts during 2017. Whilst most tweets are not Islamophobic, weak Islamophobia is considerably more prevalent (36,963 tweets) than strong (14,895 tweets). Our main input feature is a gloVe word embeddings model trained on a newly collected corpus of 140 million tweets. It outperforms a generic word embeddings model by 5.9 percentage points, demonstrating the importan4ce of context. Unexpectedly, we also find that a one-against-one multi class SVM outperforms a deep learning algorithm.
Deep Learning Framework for Wireless Systems: Applications to Optical Wireless Communications
Lee, Hoon, Lee, Sang Hyun, Quek, Tony Q. S., Lee, Inkyu
Optical wireless communication (OWC) is a promising technology for future wireless communications owing to its potentials for cost-effective network deployment and high data rate. There are several implementation issues in the OWC which have not been encountered in radio frequency wireless communications. First, practical OWC transmitters need an illumination control on color, intensity, and luminance, etc., which poses complicated modulation design challenges. Furthermore, signal-dependent properties of optical channels raise non-trivial challenges both in modulation and demodulation of the optical signals. To tackle such difficulties, deep learning (DL) technologies can be applied for optical wireless transceiver design. This article addresses recent efforts on DL-based OWC system designs. A DL framework for emerging image sensor communication is proposed and its feasibility is verified by simulation. Finally, technical challenges and implementation issues for the DL-based optical wireless technology are discussed.
Learning to Learn from Noisy Labeled Data
Li, Junnan, Wong, Yongkang, Zhao, Qi, Kankanhalli, Mohan
Abstract--Despite the success of deep neural networks (DNNs) in image classification tasks, the human-level performance relies on massive training data with high-quality manual annotations, which are expensive and time-consuming to collect. There exist many inexpensive data sources on the web, but they tend to contain inaccurate labels. Training on noisy labeled datasets causes performance degradation because DNNs can easily overfit to the label noise. To overcome this problem, we propose a noisetolerant trainingalgorithm, where a meta-learning update is performed prior to conventional gradient update. The proposed meta-learning method simulates actual training by generating synthetic noisy labels, and train the model such that after one gradient update using each set of synthetic noisy labels, the model does not overfit to the specific noise. We conduct extensive experiments on the noisy CIFAR-10 dataset and the Clothing1M dataset. The results demonstrate the advantageous performance of the proposed method compared to several state-of-the-art baselines. I. INTRODUCTION One of the key reasons why deep neural networks (DNNs) have been so successful in image classification is the collections ofmassive labeled datasets such as ImageNet [19] and COCO [14].
Recent Advances in Autoencoder-Based Representation Learning
Tschannen, Michael, Bachem, Olivier, Lucic, Mario
Learning useful representations with little or no supervision is a key challenge in artificial intelligence. We provide an in-depth review of recent advances in representation learning with a focus on autoencoder-based models. To organize these results we make use of meta-priors believed useful for downstream tasks, such as disentanglement and hierarchical organization of features. In particular, we uncover three main mechanisms to enforce such properties, namely (i) regularizing the (approximate or aggregate) posterior distribution, (ii) factorizing the encoding and decoding distribution, or (iii) introducing a structured prior distribution. While there are some promising results, implicit or explicit supervision remains a key enabler and all current methods use strong inductive biases and modeling assumptions. Finally, we provide an analysis of autoencoder-based representation learning through the lens of rate-distortion theory and identify a clear tradeoff between the amount of prior knowledge available about the downstream tasks, and how useful the representation is for this task.