Deep neural networks are traditionally trained using human-designed stochastic optimization algorithms, such as SGD and Adam. Recently, the approach of learning to optimize network parameters has emerged as a promising research topic. However, these learned black-box optimizers sometimes do not fully utilize the experience in human-designed optimizers, therefore have limitation in generalization ability. In this paper, a new optimizer, dubbed as \textit{HyperAdam}, is proposed that combines the idea of "learning to optimize" and traditional Adam optimizer. Given a network for training, its parameter update in each iteration generated by HyperAdam is an adaptive combination of multiple updates generated by Adam with varying decay rates. The combination weights and decay rates in HyperAdam are adaptively learned depending on the task. HyperAdam is modeled as a recurrent neural network with AdamCell, WeightCell and StateCell. It is justified to be state-of-the-art for various network training, such as multilayer perceptron, CNN and LSTM.

Scalable Bayesian sampling is playing an important role in modern machine learning, especially in the fast-developed unsupervised-(deep)-learning models. While tremendous progresses have been achieved via scalable Bayesian sampling such as stochastic gradient MCMC (SG-MCMC) and Stein variational gradient descent (SVGD), the generated samples are typically highly correlated. Moreover, their sample-generation processes are often criticized to be inefficient. In this paper, we propose a novel self-adversarial learning framework that automatically learns a conditional generator to mimic the behavior of a Markov kernel (transition kernel). High-quality samples can be efficiently generated by direct forward passes though a learned generator. Most importantly, the learning process adopts a self-learning paradigm, requiring no information on existing Markov kernels, e.g., knowledge of how to draw samples from them. Specifically, our framework learns to use current samples, either from the generator or pre-provided training data, to update the generator such that the generated samples progressively approach a target distribution, thus it is called self-learning. Experiments on both synthetic and real datasets verify advantages of our framework, outperforming related methods in terms of both sampling efficiency and sample quality.

Machine learning applications in medical imaging are frequently limited by the lack of quality labeled data. In this paper, we explore the self training method, a form of semi-supervised learning, to address the labeling burden. By integrating reinforcement learning, we were able to expand the application of self training to complex segmentation networks without any further human annotation. The proposed approach, reinforced self training (ReST), fine tunes a semantic segmentation networks by introducing a policy network that learns to generate pseudolabels. We incorporate an expert demonstration network, based on inverse reinforcement learning, to enhance clinical validity and convergence of the policy network. The model was tested on a pulmonary nodule segmentation task in chest X-rays and achieved the performance of a standard U-Net while using only 50% of the labeled data, by exploiting unlabeled data. When the same number of labeled data was used, a moderate to significant cross validation accuracy improvement was achieved depending on the absolute number of labels used.

Integrated gradients are widely employed to evaluate the contribution of input features in classification models because it satisfies the axioms for attribution of prediction. This method, however, requires an appropriate baseline for reliable determination of the contributions. We propose a compensated integrated gradients method that does not require a baseline. In fact, the method compensates the attributions calculated by integrated gradients at an arbitrary baseline using Shapley sampling. We prove that the method retrieves reliable attributions if the processes of input features in a classifier are mutually independent, and they are identical like shared weights in convolutional neural networks. Using three electroencephalogram datasets, we experimentally demonstrate that the attributions of the proposed method are more reliable than those of the original integrated gradients, and its computational complexity is much lower than that of Shapley sampling.

In discussions hosted on discussion forums for Massive Online Open Courses (MOOCs), references to online learning resources are often of central importance. However they are usually mentioned in free text, without appropriate hyperlinking to their associated resource. Automated learning resource mention hyperlinking and categorization will facilitate discussion and searching within MOOC forums, and also benefit the contextualization of such resources across disparate views. We propose the novel problem of learning resource mention identification inMOOC forums; i.e., to identify resource mentions in discussions, and classify them into predefined resource types. As this is a novel task with no publicly available data, we first contribute a large-scale labeled dataset - dubbed the Forum Resource Mention (FoRM) dataset - to facilitate our current research and future research on this task. FoRM contains over 10, 000 real-world forum threads in collaboration with Coursera, with more than 23, 000 manually labeled resource mentions. We then formulate this task as a sequence tagging problem and investigate solutionarchitectures to address the problem. Corresponding author Email address: peterpan10211020@gmail.com (Liangming Pan) Preprint submitted to Elsevier November 22, 2018 two major challenges that hinder the application of sequence tagging models tothe task: (1) the diversity of resource mention expression, and (2) long-range contextual dependencies. We address these challenges by incorporating character-leveland thread context information into a LSTM-CRF model. First, we incorporate a character encoder to address the out-ofvocabulary problemcaused by the diversity of mention expressions. Second, to address the context dependency challenge, we encode thread contexts using anRNN-based context encoder, and apply the attention mechanism to selectively leverage useful context information during sequence tagging. Experiments onFoRM show that the proposed method improves the baseline deep sequence tagging models notably, significantly bettering performance on instances that exemplify the two challenges.

Jewelry has been an integral part of human culture since ages. One of the most popular styles of jewelry is created by putting together precious and semi-precious stones in diverse patterns. While technology is finding its way in the production process of such jewelry, designing it remains a time-consuming and involved task. In this paper, we propose a unique approach using optimization methods coupled with machine learning techniques to generate novel stone jewelry designs at scale. Our evaluation shows that designs generated by our approach are highly likeable and visually appealing.

In this talk, Kai-Ful Lee will talk about the four waves of Artificial Intelligence (AI), and how AI will permeate every part of our lives in the next decade. He will also talk about how this will be different from previous technology revolutions -- it will be faster and be driven by not one superpower, but two (US and China). AI will add $16 trillion to our global GDP, but also cause many challenges that will be hard to solve. Kai-Fu Lee will talk in particular about AI replacing routine jobs -- the consequences, the proposed solutions that don't work (such as UBI), and end with a blueprint of co-existence between humans and AI.

Autonomous vehicle technology is advancing rapidly, and hard-core promoters contend that driverless cars could soon be the norm rather than the exception. Many other knowledgeable analysts, however, say widespread adoption of fully autonomous cars is many years -- perhaps decades -- away. The chief reason for the delay is the years it will take to generate the vast amount of data required to make self-driving cars fully safe. But whenever it finally takes over, driverless technology will do much more than ease daily commutes: It will also have a profound impact on the world's economy, notes Lawrence Burns, a former corporate vice president of research, development and planning for General Motors who supervised and encouraged GM's development of robotic driving technology. His new book with Christopher Shulgan is titled, Autonomy: The Quest to Build the Driverless Car -- And How It Will Reshape Our World. He joined the Knowledge@Wharton show on SiriusXM to talk about how a driverless world will map out. An edited transcript of the conversation follows.

Artificial Intelligence technologies and capabilities are driving digital transformation, growth, and opportunity in nearly every sector. In fact, a report by AlphaBeta urges Australia to double its pace of artificial intelligence and robotics automation to reap a $2.2 trillion market opportunity by 2030. So, why is it that AI can have such a massive impact, and should your organisation jump on the bandwagon? The short answer is that the window of competitive advantage will be small, and if you don't jump through it, one of your competitors will. There's much to be gained by using AI to improve business outcomes.

Should we be scared of artificial intelligence (AI)? Some notable individuals such as legendary physicist Stephen Hawking and Tesla and SpaceX leader and innovator Elon Musk suggest AI could potentially be very dangerous; Musk at one point was comparing AI to the dangers of the dictator of North Korea. Microsoft co-founder Bill Gates also believes there's reason to be cautious, but that the good can outweigh the bad if managed properly. Since recent developments have made super-intelligent machines possible much sooner than initially thought, the time is now to determine what dangers artificial intelligence poses. What is applied and generalized artificial intelligence?