We discuss training techniques, objectives and metrics toward mass personalization of deep learning models. In machine learning, personalization refers to the fact that every trained model should be targeted towards an individual by optimizing one or several performance metrics and often obeying additional constraints. We investigate three methods for personalized training of neural networks. They constitute three forms of curriculum learning. The methods are partially inspired by the "shaping" concept from psychology. Interestingly, we discover that extensive exposure to a limited set of training data in terms of class diversity \emph{early} in the training can lead to an irreversible reduction of the capability of a network to learn from more diverse training data. This is in close alignment with existing theories in human development. In contrast, training on a small data set covering all classes \emph{early} in the training can lead to better performance.

We present a generative model that is defined on finite sets of exchangeable, potentially high dimensional, data. As the architecture is an extension of RealNVPs, it inherits all its favorable properties, such as being invertible and allowing for exact log-likelihood evaluation. We show that this architecture is able to learn finite non-i.i.d. set data distributions, learn statistical dependencies between entities of the set and is able to train and sample with variable set sizes in a computationally efficient manner. Experiments on 3D point clouds show state-of-the art likelihoods.

Cheolhyeong Kim 1, Haeseong Moon 2, and Hyung Ju Hwang † 1 1 Department of Mathematics, POSTECH 2 Department of Mathematics, University of California, San Diego Abstract Learning graph-structured data with graph neural networks (GNNs) has been recently emerging as an important field because of its wide applicability in bioinformatics, chemoinformatics, social network analysis and data mining. Recent GNN algorithms are based on neural message passing, which enables GNNs to integrate local structures and node features recursively. However, past GNN algorithms based on 1-hop neighborhood neural message passing are exposed to a risk of loss of information on local structures and relationships. In this paper, we propose Neighborhood Edge AggregatoR (NEAR), a novel framework that aggregates relations between the nodes in the neighborhood via edges. NEAR, which can be orthogonally combined with previous GNN algorithms, gives integrated information that describes which nodes in the neighborhood are connected. Therefore, GNNs combined with NEAR reflect each node's local structure beyond the nodes themselves. Experimental results on multiple graph classification tasks show that our algorithm achieves state-of-the-art results. 1 Introduction Interest in learning graph structured data has risen rapidly in recent years because of its wide applicability in bioinformatics, chemoinformatics, social network analysis and data mining. For learning graph-structured data, we need an algorithm that can effectively represent the graph structure and relations between the graph nodes.

Fine-tuning a deep network trained with the standard cross-entropy loss is a strong baseline for few-shot learning. When fine-tuned transductively, this outperforms the current state-of-the-art on standard datasets such as Mini-Imagenet, Tiered-Imagenet, CIFAR-FS and FC-100 with the same hyper-parameters. The simplicity of this approach enables us to demonstrate the first few-shot learning results on the Imagenet-21k dataset. We find that using a large number of meta-training classes results in high few-shot accuracies even for a large number of test classes. We do not advocate our approach as the solution for few-shot learning, but simply use the results to highlight limitations of current benchmarks and few-shot protocols. We perform extensive studies on benchmark datasets to propose a metric that quantifies the "hardness" of a test episode. This metric can be used to report the performance of few-shot algorithms in a more systematic way.

The framework of information theoretic learning (ITL) has been verified as a powerful approach for robust machine learning, which improves robustness significantly in regression, feature extraction, dimensionality reduction and so on. Nevertheless, few studies utilize ITL for robust classification. In this study, we attempt to improve the robustness of the logistic regression, a fundamental method in classification, through analyzing the characteristic when the model is affected by outliers. We propose an ITL-based variant that learns by the error distribution, the performance of which is experimentally evaluated on two toy examples and several public datasets, compared with two traditional methods and two states of the art. The results demonstrate that the novel method can outperform the states of the art evidently in some cases, and behaves with desirable potential to achieve better robustness in complex situations than existing methods.

Author profiling is the characterization of an author through some key attributes such as gender, age, and language. In this paper, a RNN model with Attention (RNNwA) is proposed to predict the gender of a twitter user using their tweets. Both word level and tweet level attentions are utilized to learn 'where to look'. This model (https://github.com/Darg-Iztech/gender-prediction-from-tweets) is improved by concatenating LSA-reduced n-gram features with the learned neural representation of a user. Both models are tested on three languages: English, Spanish, Arabic. The improved version of the proposed model (RNNwA + n-gram) achieves state-of-the-art performance on English and has competitive results on Spanish and Arabic.

Entity recommendation, providing search users with an improved experience via assisting them in finding related entities for a given query, has become an indispensable feature of today's search engines. Existing studies typically only consider the queries with explicit entities. They usually fail to handle complex queries that without entities, such as "what food is good for cold weather", because their models could not infer the underlying meaning of the input text. In this work, we believe that contexts convey valuable evidence that could facilitate the semantic modeling of queries, and take them into consideration for entity recommendation. In order to better model the semantics of queries and entities, we learn the representation of queries and entities jointly with attentive deep neural networks. We evaluate our approach using large-scale, real-world search logs from a widely used commercial Chinese search engine. Our system has been deployed in ShenMa Search Engine and you can fetch it in UC Browser of Alibaba. Results from online A/B test suggest that the impression efficiency of click-through rate increased by 5.1% and page view increased by 5.5%.

This paper is about recovering the unseen future data from a given sequence of historical samples, so called as \emph{future data recovery}---a significant problem closely related to time series forecasting. To address the problem, it is now prevalent to use deep neural networks, which are actually built upon the hypothesis that the desired evolution law can be learnt by using many observed samples to feed an overparameterized network. In practice, however, it is not always feasible to obtain a huge mass of training samples. To overcome the issue, we would suggest to consider a different methodology. Namely, we convert future data recovery into a more inclusive problem called \emph{sequential tensor completion} (STC), which is to restore a latent tensor of sequential structure from a sampling of its entries. Unlike the ordinary tensor completion problem studied in the majority of literature, STC has a distinctive setup that allows the locations of missing entries to be distributed arbitrarily, integrating seamlessly the future values of time series into the framework of missing data. Then we propose two methods to address STC, including Discrete Fourier Transform based $\ell_1$ minimization ($\mathrm{DFT}_{\ell_1}$) and Convolution Nuclear Norm Minimization (CNNM). We provide theoretical results to guarantee the recovery performance of the proposed methods. Remarkably, our theories disclose an important message; that is, under certain conditions, the unseen future values are indeed recoverable from the historical observations. Experiments on univariate time series, images and videos show encouraging results.

Aljosa Smolic 1 smolica@scss.tcd.ie 1 V-SENSE School of Computer Science and Statistics T rinity College Dublin, Ireland 2 University of Houston-Victoria, Victoria, T exas, US 3 CAAD Department of Architecture ETH, Zurich, Switzerland 4 Department of Architecture T arbiat Modares University T ehran, Iran Abstract Scene understanding of full-scale 3D models of an urban area remains a challenging task. While advanced computer vision techniques offer cost-effective approaches to analyse 3D urban elements, a precise and densely labelled dataset is quintessential. The paper presents the first-ever labelled dataset for a highly dense Aerial Laser Scanning (ALS) point cloud at city-scale. This work introduces a novel benchmark dataset that includes a manually annotated point cloud for over 260 million laser scanning points into 100'000 (approx.) Objects are labelled into 13 classes using hierarchical levels of detail from large ( i.e. building, vegetation and ground) to refined ( i.e. window, door and tree) elements. To validate the performance of our dataset, two different applications are showcased. Firstly, the labelled point cloud is employed for training Convolutional Neural Networks (CNNs) to classify urban elements. The dataset is tested on the well-known state-of-the-art CNNs ( i.e. Secondly, the complete ALS dataset is applied as detailed ground truth for city-scale image-based 3D reconstruction. 1 Introduction In computer vision, automated identification of three-dimensional (3D) assets in an unstructured large dataset is essential for scene understanding.

This paper presents entropy maps, an approach to describing and visualising uncertainty among alternative potential movement intentions in pedestrian simulation models. In particular, entropy maps show the instantaneous level of randomness in decisions of a pedestrian agent situated in a specific point of the simulated environment with an heatmap approach. Experimental results highlighting the relevance of this tool supporting modelers are provided and discussed. Keywords: Data Visualization · Modelling and Simulation · Stochastic Models. 1 Introduction & Related Works Computer simulation of complex systems often employs stochastic models: implied randomness is a way to account for aspects that are potentially relevant to the overall phenomenon but cannot be explicitly considered to keep the model and the modelling phase manageable [3]. Pedestrian and crowd behaviour simulation, for instance, requires considering different kinds of decisions, taken at distinct levels of abstraction, employing heterogeneous information and knowledge about the environment, from path planning [7] to the regulation of distance from other pedestrians and obstacles present in the environment[2,8]. Exploring implications of randomness and situations of indecision, irresolution in case of choice among alternative lines of behaviour such as the exits from an environment in an emergency situation [10], can be a very significant step, with important implications of overall simulation results. This paper presents an approach to describing and visualising uncertainty among alternative potential movement intentions in pedestrian simulation models. As in the framework of probability theory [12], we use the concept of entropy to provide a measure of uncertainty over the simulated space The paper, first of all, describes a general decision making model for supporting wayfinding, which comes from previous work by the authors [8,7].