Goto

Collaborating Authors

Neural Architecture Search in Embedding Space

arXiv.org Machine Learning

The neural architecture search (NAS) algorithm with reinforcement learning can be a powerful and novel framework for the automatic discovering process of neural architectures. However, its application is restricted by noncontinuous and high-dimensional search spaces, which result in difficulty in optimization. To resolve these problems, we proposed NAS in embedding space (NASES), which is a novel framework. Unlike other NAS with reinforcement learning approaches that search over a discrete and high-dimensional architecture space, this approach enables reinforcement learning to search in an embedding space by using architecture encoders and decoders. The current experiment demonstrated that the performance of the final architecture network using the NASES procedure is comparable with that of other popular NAS approaches for the image classification task on CIFAR-10. The beneficial-performance and effectiveness of NASES was impressive even when only the architecture-embedding searching and pre-training controller were applied without other NAS tricks such as parameter sharing. Specifically, considerable reduction in searches was achieved by reducing the average number of searching to 100 architectures to achieve a final architecture for the NASES procedure. Introduction Deep neural networks have enabled advances in image recognition, sequential pattern recognition, recommendation systems, and various tasks in the past decades.


Concepts of Advanced Deep Learning Architectures

#artificialintelligence

Deep Learning algorithms consist of a different set of models due to the flexibility that neural network allows while building a full fledged end-to-end model. Computer vision is basically based on the theoretical and technological aspect for building artificial systems which have the ability to gather automatic visual information from images or multi-dimensional data. It is focussed on the self-executing extraction, analysis and studying about useful information from a particular image or a sequence of images. Broadly the computer vision consists of tasks like Object Recognition, Identification, Detection, Content-based image retrieval, Image Segmentation and much more. After getting an insight of what basically advanced architecture is and computer vision we move towards the study of some important deep learning advanced architecture.


Auto-GNN: Neural Architecture Search of Graph Neural Networks

arXiv.org Machine Learning

Graph neural networks (GNN) has been successfully applied to operate on the graph-structured data. Given a specific scenario, rich human expertise and tremendous laborious trials are usually required to identify a suitable GNN architecture. It is because the performance of a GNN architecture is significantly affected by the choice of graph convolution components, such as aggregate function and hidden dimension. Neural architecture search (NAS) has shown its potential in discovering effective deep architectures for learning tasks in image and language modeling. However, existing NAS algorithms cannot be directly applied to the GNN search problem. First, the search space of GNN is different from the ones in existing NAS work. Second, the representation learning capacity of GNN architecture changes obviously with slight architecture modifications. It affects the search efficiency of traditional search methods. Third, widely used techniques in NAS such as parameter sharing might become unstable in GNN. To bridge the gap, we propose the automated graph neural networks (AGNN) framework, which aims to find an optimal GNN architecture within a predefined search space. A reinforcement learning based controller is designed to greedily validate architectures via small steps. AGNN has a novel parameter sharing strategy that enables homogeneous architectures to share parameters, based on a carefully-designed homogeneity definition. Experiments on real-world benchmark datasets demonstrate that the GNN architecture identified by AGNN achieves the best performance, comparing with existing handcrafted models and tradistional search methods.


CNN Architectures, a Deep-dive

#artificialintelligence

VGG Net is a plain and straight forward CNN architecture among all other. Thought it looks simple, it do outperform many complex architectures. It is the 1st runner-up in ImageNet Challenge in 2014. As shown above, there are totally 6 VGGNet Architectures. Among them, VGG-16 and VGG-19 are popular.


Utility Analysis of Network Architectures for 3D Point Cloud Processing

arXiv.org Machine Learning

Note that most widely used benchmark datasets for point cloud classification only contain foreground objects. Therefore, we generate a new dataset, where each point cloud contains both the foreground object and the background. In this new dataset, the background is composed of points that carry no relevant information of the foreground. We will introduce details in Section 5. Metric 3, rotation robustness: The rotation robustness is proposed to measure whether a DNN uses similar subsets of two point clouds to compute the intermediate-layer feature, if the two point clouds have the same shape but different orientations. Let X θ 1 and X θ 2 denote the point clouds that have the same global shape but different orientations θ 1 and θ 2. To quantify the similarity of the attention on the two point clouds, we compute the Jensen-Shannon divergence between the distributions of the perturbed inputs ˆ X θ 1 X θ 1 δ 1 and ˆ X θ 2 X θ 2 δ 2. ˆ X θ 1 and ˆ X θ 2 denote the perturbed inputs, which are computed to measure information discarding in Equation (1).