Deep Learning
Key Phrase Extraction & Applause Prediction
Yadav, Krishna, Choudhary, Lakshya
With the increase in content availability over the internet it is very difficult to get noticed. It has become an upmost the priority of the blog writers to get some feedback over their creations to be confident about the impact of their article. We are training a machine learning model to learn popular article styles, in the form of vector space representations using various word embeddings, and their popularity based on claps and tags.
Iranis: A Large-scale Dataset of Farsi License Plate Characters
Tourani, Ali, Soroori, Sajjad, Shahbahrami, Asadollah, Akoushideh, Alireza
Providing huge amounts of data is a fundamental demand when dealing with Deep Neural Networks (DNNs). Employing these algorithms to solve computer vision problems resulted in the advent of various image datasets to feed the most common visual imagery deep structures, known as Convolutional Neural Networks (CNNs). In this regard, some datasets can be found that contain hundreds or even thousands of images for license plate detection and optical character recognition purposes. However, no publicly available image dataset provides such data for the recognition of Farsi characters used in car license plates. The gap has to be filled due to the numerous advantages of developing accurate deep learning-based systems for law enforcement and surveillance purposes. This paper introduces a large-scale dataset that includes images of numbers and characters used in Iranian car license plates. The dataset, named Iranis, contains more than 83,000 images of Farsi numbers and letters collected from real-world license plate images captured by various cameras. The variety of instances in terms of camera shooting angle, illumination, resolution, and contrast make the dataset a proper choice for training DNNs. Dataset images are manually annotated for object detection and image classification. Finally, and to build a baseline for Farsi character recognition, the paper provides a performance analysis using a YOLO v.3 object detector.
Multi-shot NAS for Discovering Adversarially Robust Convolutional Neural Architectures at Targeted Capacities
Ning, Xuefei, Zhao, Junbo, Li, Wenshuo, Zhao, Tianchen, Yang, Huazhong, Wang, Yu
Convolutional neural networks (CNNs) are vulnerable to adversarial examples, and studies show that increasing the model capacity of an architecture topology (e.g., width expansion) can bring consistent robustness improvements. This reveals a clear robustness-efficiency trade-off that should be considered in architecture design. Recent studies have employed one-shot neural architecture search (NAS) to discover adversarially robust architectures. However, since the capacities of different topologies cannot be easily aligned during the search process, current one-shot NAS methods might favor topologies with larger capacity in the supernet. And the discovered topology might be sub-optimal when aligned to the targeted capacity. This paper proposes a novel multi-shot NAS method to explicitly search for adversarially robust architectures at a certain targeted capacity. Specifically, we estimate the reward at the targeted capacity using interior extra-polation of the rewards from multiple supernets. Experimental results demonstrate the effectiveness of the proposed method. For instance, at the targeted FLOPs of 1560M, the discovered MSRobNet-1560 (clean 84.8%, PGD100 52.9%) outperforms the recent NAS-discovered architecture RobNet-free (clean 82.8%, PGD100 52.6%) with similar FLOPs. Codes are available at https://github.com/walkerning/aw_nas.
Neural Architecture Search via Combinatorial Multi-Armed Bandit
Huang, Hanxun, Ma, Xingjun, Erfani, Sarah M., Bailey, James
Neural Architecture Search (NAS) has gained significant popularity as an effective tool for designing high performance deep neural networks (DNNs). NAS can be performed via policy gradient, evolutionary algorithms, differentiable architecture search or tree-search methods. While significant progress has been made for both policy gradient and differentiable architecture search, tree-search methods have so far failed to achieve comparable accuracy or search efficiency. In this paper, we formulate NAS as a Combinatorial Multi-Armed Bandit (CMAB) problem (CMAB-NAS). This allows the decomposition of a large search space into smaller blocks where tree-search methods can be applied more effectively and efficiently. We further leverage a tree-based method called Nested Monte-Carlo Search to tackle the CMAB-NAS problem. On CIFAR-10, our approach discovers a cell structure that achieves a low error rate that is comparable to the state-of-the-art, using only 0.58 GPU days, which is 20 times faster than current tree-search methods. Moreover, the discovered structure transfers well to large-scale datasets such as ImageNet.
An iterative K-FAC algorithm for Deep Learning
Kronecker-factored Approximate Curvature (K-FAC) method is a high efficiency second order optimizer for the deep learning. Its training time is less than SGD(or other first-order method) with same accuracy in many large-scale problems. The key of K-FAC is to approximates Fisher information matrix (FIM) as a block-diagonal matrix where each block is an inverse of tiny Kronecker factors. In this short note, we present CG-FAC -- an new iterative K-FAC algorithm. It uses conjugate gradient method to approximate the nature gradient. This CG-FAC method is matrix-free, that is, no need to generate the FIM matrix, also no need to generate the Kronecker factors A and G. We prove that the time and memory complexity of iterative CG-FAC is much less than that of standard K-FAC algorithm.
Deep learning-based prediction of piled-up status and payload distribution of bulk material
The piled-up status of bulk material in a haul truck body determines the load balance, hence affects the mining operations’ efficiency. Prediction of Piled-up Status and Payload Distribution (PSPD) of bulk material contributes to providing optimal dumping positions to improve the vehicle’s stress state and service life. This work introduces a novel deep learning-based PSPD prediction method from images. A two-stage prediction-regression CNN model is designed to automatically extract image features to obtain the PSPD of the current state. The PSPD prediction is accomplished via a backward-propagation neural network (BPNN).
Understand TensorFlow Basic with Python
Why everyone wants to learn TensorFlow in deep learning, as we deep dive into the machine learning projects, we use the "sklearn" library, and when we talk about deep neural networks, TensorFlow comes into the picture. Anaconda distribution is perfect for data science and machine learning with the pre-installed library packages, but we have to install TensorFlow explicitly because it does not come in anaconda distribution. To download the TensorFlow write the following command in the anaconda prompt and press enter. TensorFlow is a library for deep computational processes in many applications and is widely used in deep neural networks. Before knowing what tensor is, go through with some definitions.
Timnit Gebru's Exit From Google Exposes a Crisis in AI
Alex Hanna is a sociologist and senior research scientist on the Ethical AI team at Google. Meredith Whittaker is the Minderoo Research Professor at NYU, the faculty director of the AI Now Institute at NYU, and a long-time tech worker who helped lead labor organizing at Google. This year has held many things, among them bold claims of artificial intelligence breakthroughs. Industry commentators speculated that the language-generation model GPT-3 may have achieved "artificial general intelligence," while others lauded Alphabet subsidiary DeepMind's protein-folding algorithm--Alphafold--and its capacity to "transform biology." While the basis of such claims is thinner than the effusive headlines, this hasn't done much to dampen enthusiasm across the industry, whose profits and prestige are dependent on AI's proliferation.
The Future is Here! Have You Checked OpenAI's GPT-3 Yet?
Ever wonder how close has AI gotten to impersonating human beings? The latest GPT-3 can code computer programs, compose tweets, summarize emails, write news, answer questions, translate languages, and write fiction and poetry too. It can take up almost any virtual English language task. In the latest recent milestone on YouTube, it created an app that functions similar to Instagram. Dubbed as one of the most important advancements in AI in recent years, GPT-3 or Generative Pre-Trained Transformer 3 has raised the AI goal posts many notches toward the stratosphere.
Relational Deep Reinforcement Learning for Routing in Wireless Networks
Manfredi, Victoria, Wolfe, Alicia, Wang, Bing, Zhang, Xiaolan
While routing in wireless networks has been studied extensively, existing protocols are typically designed for a specific set of network conditions and so cannot accommodate any drastic changes in those conditions. For instance, protocols designed for connected networks cannot be easily applied to disconnected networks. In this paper, we develop a distributed routing strategy based on deep reinforcement learning that generalizes to diverse traffic patterns, congestion levels, network connectivity, and link dynamics. We make the following key innovations in our design: (i) the use of relational features as inputs to the deep neural network approximating the decision space, which enables our algorithm to generalize to diverse network conditions, (ii) the use of packet-centric decisions to transform the routing problem into an episodic task by viewing packets, rather than wireless devices, as reinforcement learning agents, which provides a natural way to propagate and model rewards accurately during learning, and (iii) the use of extended-time actions to model the time spent by a packet waiting in a queue, which reduces the amount of training data needed and allows the learning algorithm to converge more quickly. We evaluate our routing algorithm using a packet-level simulator and show that the policy our algorithm learns during training is able to generalize to larger and more congested networks, different topologies, and diverse link dynamics. Our algorithm outperforms shortest path and backpressure routing with respect to packets delivered and delay per packet.