Education
VecHGrad for solving accurately complex tensor decomposition
Charlier, Jeremy, Ormazabal, Gaston, State, Radu, Hilger, Jean
Tensor decomposition, a collection of factorization techniques for multidimensional arrays, are among the most general and powerful tools for scientific analysis. However, because of their increasing size, today's data sets require more complex tensor decomposition involving factorization with multiple matrices and diagonal tensors such as DEDICOM or PARATUCK2. Traditional tensor resolution algorithms such as Stochastic Gradient Descent (SGD), Non-linear Conjugate Gradient descent (NCG) or Alternating Least Square (ALS), cannot be easily applied to complex tensor decomposition or often lead to poor accuracy at convergence. We propose a new resolution algorithm, called VecHGrad, for accurate and efficient stochastic resolution over all existing tensor decomposition, specifically designed for complex decomposition. VecHGrad relies on gradient, Hessian-vector product and adaptive line search to ensure the convergence during optimization. Our experiments on five real-world data sets with the state-of-the-art deep learning gradient optimization models show that VecHGrad is capable of converging considerably faster because of its superior theoretical convergence rate per step. Therefore, VecHGrad targets as well deep learning optimizer algorithms. The experiments are performed for various tensor decomposition including CP, DEDICOM and PARATUCK2. Although it involves a slightly more complex update rule, VecHGrad's runtime is similar in practice to that of gradient methods such as SGD, Adam or RMSProp.
Cold Case: The Lost MNIST Digits
Although the popular MNIST dataset [LeCun et al., 1994] is derived from the NIST database [Grother and Hanaoka, 1995], the precise processing steps for this derivation have been lost to time. We propose a reconstruction that is accurate enough to serve as a replacement for the MNIST dataset, with insignificant changes in accuracy. We trace each MNIST digit to its NIST source and its rich metadata such as writer identifier, partition identifier, etc. We also reconstruct the complete MNIST test set with 60,000 samples instead of the usual 10,000. Since the balance 50,000 were never distributed, they enable us to investigate the impact of twenty-five years of MNIST experiments on the reported testing performances. Our results unambiguously confirm the trends observed by Recht et al. [2018, 2019]: although the misclassification rates are slightly off, classifier ordering and model selection remain broadly reliable. We attribute this phenomenon to the pairing benefits of comparing classifiers on the same digits.
Learning to Identify High Betweenness Centrality Nodes from Scratch: A Novel Graph Neural Network Approach
Fan, Changjun, Zeng, Li, Ding, Yuhui, Chen, Muhao, Sun, Yizhou, Liu, Zhong
Betweenness centrality (BC) is one of the most used centrality measures for network analysis, which seeks to describe the importance of nodes in a network in terms of the fraction of shortest paths that pass through them. It is key to many valuable applications, including community detection and network dismantling. Computing BC scores on large networks is computationally challenging due to high time complexity. Many approximation algorithms have been proposed to speed up the estimation of BC, which are mainly sampling-based. However, these methods are still prone to considerable execution time on large-scale networks, and their results are often exacerbated when small changes happen to the network structures. In this paper, we focus on identifying nodes with high BC in a graph, since many application scenarios are built upon retrieving nodes with top-k BC. Different from previous heuristic methods, we turn this task into a learning problem and design an encoder-decoder based framework to resolve the problem. More specifcally, the encoder leverages the network structure to encode each node into an embedding vector, which captures the important structural information of the node. The decoder transforms the embedding vector for each node into a scalar, which captures the relative rank of this node in terms of BC. We use the pairwise ranking loss to train the model to identify the orders of nodes regarding their BC. By training on small-scale networks, the learned model is capable of assigning relative BC scores to nodes for any unseen networks, and thus identifying the highly-ranked nodes. Comprehensive experiments on both synthetic and real-world networks demonstrate that, compared to representative baselines, our model drastically speeds up the prediction without noticeable sacrifce in accuracy, and outperforms the state-of-the-art by accuracy on several large real-world networks.
Edge Intelligence: Paving the Last Mile of Artificial Intelligence with Edge Computing
Zhou, Zhi, Chen, Xu, Li, En, Zeng, Liekang, Luo, Ke, Zhang, Junshan
With the breakthroughs in deep learning, the recent years have witnessed a booming of artificial intelligence (AI) applications and services, spanning from personal assistant to recommendation systems to video/audio surveillance. More recently, with the proliferation of mobile computing and Internet-of-Things (IoT), billions of mobile and IoT devices are connected to the Internet, generating zillions Bytes of data at the network edge. Driving by this trend, there is an urgent need to push the AI frontiers to the network edge so as to fully unleash the potential of the edge big data. To meet this demand, edge computing, an emerging paradigm that pushes computing tasks and services from the network core to the network edge, has been widely recognized as a promising solution. The resulted new inter-discipline, edge AI or edge intelligence, is beginning to receive a tremendous amount of interest. However, research on edge intelligence is still in its infancy stage, and a dedicated venue for exchanging the recent advances of edge intelligence is highly desired by both the computer system and artificial intelligence communities. To this end, we conduct a comprehensive survey of the recent research efforts on edge intelligence. Specifically, we first review the background and motivation for artificial intelligence running at the network edge. We then provide an overview of the overarching architectures, frameworks and emerging key technologies for deep learning model towards training/inference at the network edge. Finally, we discuss future research opportunities on edge intelligence. We believe that this survey will elicit escalating attentions, stimulate fruitful discussions and inspire further research ideas on edge intelligence.
Microsoft Launches Several New Machine Learning Services and Extends Its Cognitive Services
Before its Build Developer Conference, Microsoft released several new Machine Learning services and Cognitive Services updates, ranging from no-code tools to hosted notebooks, with several new APIs and other services in-between. The Cognitive Services updates include an API for building personalization features, a form recognizer for automating data entry, a handwriting recognition API, and an enhanced speech recognition service that focuses on transcribing conversations. According to a TechCrunch article around updates for Cognitive Services, an essential service is Personalizer โ a service providing a machine learning technique that doesn't need the kind of labeled training data typically used in machine learning. Microsoft VP Scott Guthrie, leading the company's Cloud and AI Group, said in a VentureBeat article: Use of Personalizer in Microsoft stores and online systems has led to performance improvements of more than 40% in some instances. Note that users can leverage Personalizer next to existing recommendation tools on the Azure platform.
Cutting-Edge AI: Deep Reinforcement Learning in Python - Couponos
This course is going to show you a few different ways: including the powerful A2C (Advantage Actor-Critic) algorithm, the DDPG (Deep Deterministic Policy Gradient) algorithm, and evolution strategies. Evolution strategies is a new and fresh take on reinforcement learning, that kind of throws away all the old theory in favor of a more "black box" approach, inspired by biological evolution.
Online Learning Made Simple - Anytime, Anywhere Simpliv
Artificial Intelligence has come a long way from being the stuff of science fiction movies and books to becoming an integral part of our daily lives. Today, AI is one of the fastest growing global industries. Investments and experiments in AI have been taking place all around the world. Given its unimaginably wide range of uses; AI is a field of expertise that is set to grow in a very huge way over the coming years. AI professionals are among the highest paid in the field of IT. Ans: Artificial Intelligence is a part of computer science that aims to create machine that are intelligent and seek to work and react the way humans do. Q2)What to you understand by an artificial intelligence Neural Network?
AI Goes To High School
Aidan Wen is well on his way toward a career in artificial intelligence. The high school junior already has two semesters of machine-learning courses under his belt. Last summer he competed for a $12,000 prize sponsored by the Radiological Society of North America for the best ML model for spotting signs of pneumonia in lung X-rays. This year, he has entered another competition seeking a system for early detection of earthquakes using audio files. Next, he wants to try his hand at a project using natural language processing.
Enterprise Search and Machine Learning: A Match Whose Time Has Come
Over the last few years, artificial intelligence and machine learning have increasingly come up in conversations about enterprise search. As artificial intelligence (AI) and its cousin, machine learning (ML), increased in accuracy and ease of integration, instances of them being directly integrated with or running alongside of search to improve results increased as well. But chances are you remember when search relevancy was based on simple metrics like term frequency -- the document with the largest number of instances was ranked highest, and documents with fewer instances ranked lower. You were able to provide stop words like "the" and "of" whose frequent use typically added no value in retrieving relevant documents. The only content really useful to the search engine was the terms in the user query.