The influence of deep learning is continuously expanding across different domains, and its new applications are ubiquitous. The question of neural network design thus increases in importance, as traditional empirical approaches are reaching their limits. Manual design of network architectures from scratch relies heavily on trial and error, while using existing pretrained models can introduce redundancies or vulnerabilities. Automated neural architecture design is able to overcome these problems, but the most successful algorithms operate on significantly constrained design spaces, assuming the target network to consist of identical repeating blocks. We propose a probabilistic representation of a neural network structure under the assumption of independence between layer types. The probability matrix (prototype) can describe general feedforward architectures and is equivalent to the population of models, while being simple to interpret and analyze. We construct an architecture search algorithm, inspired by the estimation of distribution algorithms, to take advantage of this representation. The probability matrix is tuned towards generating high-performance models by repeatedly sampling the architectures and evaluating the corresponding networks. Our algorithm is shown to discover models which are competitive with those produced by existing architecture search methods, both in accuracy and computational costs, despite the conceptual simplicity and the comparatively limited scope of achievable designs.

A critical part of multi-person multi-camera tracking is person re-identification (re-ID) algorithm, which recognizes and retains identities of all detected unknown people throughout the video stream. Many re-ID algorithms today exemplify state of the art results, but not much work has been done to explore the deployment of such algorithms for computation and power constrained real-time scenarios. In this paper, we study the effect of using a light-weight model, MobileNet-v2 for re-ID and investigate the impact of single (FP32) precision versus half (FP16) precision for training on the server and inference on the edge nodes. We further compare the results with the baseline model which uses ResNet-50 on state of the art benchmarks including CUHK03, Market-1501, and Duke-MTMC. The MobileNet-V2 mixed precision training method can improve both inference throughput on the edge node, and training time on server $3.25\times$ reaching to 27.77fps and $1.75\times$, respectively and decreases power consumption on the edge node by $1.45\times$, while it deteriorates accuracy only 5.6\% in respect to ResNet-50 single precision on the average for three different datasets. The code and pre-trained networks are publicly available at https://github.com/TeCSAR-UNCC/person-reid.

Detecting abrupt changes in time-series data has attracted rese archers in the statistics and data mining communities for decades Basseville and Nikiforov ( 1993). Based on the instantaneousness of detection, changepoint detection algorithm s can be classified into two categories: online changepoint detection and offline changepoint de tection. While the online change detection targets on data that requires instantaneous r esponses, the offline detection algorithm often triggers delay, which leads to more accurate result s. This literature review mainly focuses on the online changepoint detection algorithms. There are plenty of changepoint detection algorithms that have be en proposed and proved pragmatic. The pioneering works Basseville and Nikiforov ( 1993) compared the probability distributions of time-series samples over the past and pr esent intervals. The algorithm demonstrates an abrupt change when two distributions a re significantly different.

Network representation learning, a fundamental research problem which aims at learning low-dimension node representations on graph-structured data, has been extensively studied in the research community. By generalizing the power of neural networks on graph-structured data, graph neural networks (GNNs) achieve superior capability in network representation learning. However, the node features of many real-world graphs could be high-dimensional and sparse, rendering the learned node representations from existing GNN architectures less expressive. The main reason lies in that those models directly makes use of the raw features of nodes as input for the message-passing and have limited power in capturing sophisticated interactions between features. In this paper, we propose a novel GNN framework for learning node representations that incorporate high-order feature interactions on feature-sparse graphs. Specifically, the proposed message aggregator and feature factorizer extract two channels of embeddings from the feature-sparse graph, characterizing the aggregated node features and high-order feature interactions, respectively. Furthermore, we develop an attentive fusion network to seamlessly combine the information from two different channels and learn the feature interaction-aware node representations. Extensive experiments on various datasets demonstrate the effectiveness of the proposed framework on a variety of graph learning tasks.

Linear and non-linear measures of heart rate variability (HRV) are widely investigated as non-invasive indicators of health. Stress has a profound impact on heart rate, and different meditation techniques have been found to modulate heartbeat rhythm. This paper aims to explore the process of identifying appropriate metrices from HRV analysis for sonification. Sonification is a type of auditory display involving the process of mapping data to acoustic parameters. This work explores the use of auditory display in aiding the analysis of HRV leveraged by unsupervised machine learning techniques. Unsupervised clustering helps select the appropriate features to improve the sonification interpretability. Vocal synthesis sonification techniques are employed to increase comprehension and learnability of the processed data displayed through sound. These analyses are early steps in building a real-time sound-based biofeedback training system.

Semi-implicit graph variational auto-encoder (SIG-VAE) is proposed to expand the flexibility of variational graph auto-encoders (VGAE) to model graph data. SIG-VAE employs a hierarchical variational framework to enable neighboring node sharing for better generative modeling of graph dependency structure, together with a Bernoulli-Poisson link decoder. Not only does this hierarchical construction provide a more flexible generative graph model to better capture real-world graph properties, but also does SIG-VAE naturally lead to semi-implicit hierarchical variational inference that allows faithful modeling of implicit posteriors of given graph data, which may exhibit heavy tails, multiple modes, skewness, and rich dependency structures. Compared to VGAE, the derived graph latent representations by SIG-VAE are more interpretable, due to more expressive generative model and more faithful inference enabled by the flexible semi-implicit construction. Extensive experiments with a variety of graph data show that SIG-VAE significantly outperforms state-of-the-art methods on several different graph analytic tasks.

Mohsen Farhadloo, PhD John Molson Scool of Business, Concordia University mohsen.farhadloo@concordia.ca August 21, 2019 Abstract In this paper we introduce score embedding, a neural network based model to learn interpretable vector representations for words. Score embedding is a supervised method that takes advantage of the labeled training data and the neural network architecture to learn interpretable representations for words. Health care has been a controversial issue between political parties in the United States. In this paper we use the discussions on Twitter regarding different issues of affordable care act to identify the public opinion about the existing health care plans using the proposed score embedding. Our results indicate our approach effectively incorporates the sentiment information and outperforms or is at least comparable to the state-of-the-art methods and the negative sentiment towards "TrumpCare" was consistently greater than neutral and positive sentiment over time. 1 Introduction Sentiment analysis as a type of text categorization is the task of identifying the sentiment orientation of documents written in natural language which assigns one of the predefined sentiment categories into a whole document or pieces of the document such as phrases or sentences [23, 8]. Many studies used binary classification and reported high performance [18, 29, 24] and some studies have observed that the performance of the categorization reduces as the number of sentiment categories increases [2, 16, 3, 11]. Bag-Of-Words (BOW), a standard approach for text categorization, represents a document by a vector that indicates the words that appear in the document.

Recent years have seen increased interest in performance guarantees of gradient descent algorithms for non-convex optimization. A number of works have uncovered that gradient noise plays a critical role in the ability of gradient descent recursions to efficiently escape saddle-points and reach second-order stationary points. Most available works limit the gradient noise component to be bounded with probability one or sub-Gaussian and leverage concentration inequalities to arrive at high-probability results. We present an alternate approach, relying primarily on mean-square arguments and show that a more relaxed relative bound on the gradient noise variance is sufficient to ensure efficient escape from saddle-points without the need to inject additional noise, employ alternating step-sizes or rely on a global dispersive noise assumption, as long as a gradient noise component is present in a descent direction for every saddle-point.

In this paper, w e have presented analogies between the regularization methods for deep learning and data augmentation process interpreted as a noise injection. It was shown that, by generating the input data from high - level features, it is possible to regularize hidden layers of the netwo rk by exploiting the ability of deep networks to learn hierarchical representations . The analysis given here is theoretical, but there already are experimental results that partially confirm these observations . A case of convolutional neural networks for stenosis detection [14] have shown that pretraining the network on artificial dataset results in reduction of test error rate on real dataset, and, thus, smaller generalization gap. An improvement of test accuracy was also observed in the case of recurrent neural networks for ECG filtering, pretrained with synthetic signals [15] . A more definitive confirmation should be expected by the comparison of models trained for the same task with dataset s created by injecting noise either into input features or high - level features of the real data.

Proposed by Freund and Schapire ( 1997), boosting is a general issue of constructing an extremely accurate prediction with numerous roughly accurate pred ictions. Addressed by Friedman ( 2001, 2002) and Natekin and Knoll ( 2013), the Gradient Boosting Machines (GBM) seeks to build predictive models through back-fittings and no n-parametric regressions. Instead of building a single model, the GBM starts by generatin g an initial model and constantly fits new models through loss function minimization to prod uce the most precise model ( Natekin and Knoll, 2013). This survey concentrates on the mathematical derivations of the gradient boosting algorithms. In Section 2, we analyze the optimization methods for par ametric and nonparametric models. Section 3 covers the definitions of different typ es of loss functions. In Section 4, we present different types of boosting algorithms, while in Section 5, we explore the combination of boosting algorithms and ranking algorithms to ran k the real-world data.