We propose Bayesian hypernetworks: a framework for approximate Bayesian inference in neural networks. A Bayesian hypernetwork, $h$, is a neural network which learns to transform a simple noise distribution, $p(\epsilon) = \mathcal{N}(0,I)$, to a distribution $q(\theta) \doteq q(h(\epsilon))$ over the parameters $\theta$ of another neural network (the "primary network"). We train $q$ with variational inference, using an invertible $h$ to enable efficient estimation of the variational lower bound on the posterior $p(\theta | \mathcal{D})$ via sampling. In contrast to most methods for Bayesian deep learning, Bayesian hypernets can represent a complex multimodal approximate posterior with correlations between parameters, while enabling cheap i.i.d. sampling of $q(\theta)$. We demonstrate these qualitative advantages of Bayesian hypernets, which also achieve competitive performance on a suite of tasks that demonstrate the advantage of estimating model uncertainty, including active learning and anomaly detection.

Count data are ubiquitous in modern statistical applications. Such data are often cross-classified into contingency tables, where iterative proportional scaling (IPS) can be applied as a standard tool (Fienberg and Meyer, 2006). IPS was firstly introduced by Deming and Stephan (1940) to adjust a contingency table to obey prescribed column and row marginals, the problem of which is referred to as matrix raking nowadays. In general, IPS can be applied to Kullback-Leibler (KL) divergence minimization with linear constraints, and Poisson log-linear model fitting on multi-way tables (Ireland and Kullback, 1968; Bishop et al., 1975), and there is an interesting 1 duality between these two types of problems (Good, 1963; Csiszár, 1975). Theoretical studies regarding the convergence properties of IPS undergo a long history and we refer to Pukelsheim (2014) for a comprehensive survey.

We begin by calculating the maximum likelihood estimates of the EVSDT parameters, separately for each participant in the data set. Before doing so, I note that this data processing is only required for manual calculation of the point estimates; the modeling methods described below take the raw data and therefore don't require this annoying step. First, we'll compute for each trial whether the participant's response was a hit, false alarm, correct rejection, or a miss. We'll do this by creating a new variable, type: Then we can simply count the numbers of these four types of trials for each participant, and put the counts on one row per participant. For a single subject, d' can be calculated as the difference of the standardized hit and false alarm rates (Stanislaw and Todorov 1999): Its inverse, \(\Phi {-1}\), converts a proportion (such as a hit rate or false alarm rate) into a z score.

We prove near-tight concentration of measure for polynomial functions of the Ising model under high temperature. For any degree $d$, we show that a degree-$d$ polynomial of a $n$-spin Ising model exhibits exponential tails that scale as $\exp(-r^{2/d})$ at radius $r=\tilde{\Omega}_d(n^{d/2})$. Our concentration radius is optimal up to logarithmic factors for constant $d$, improving known results by polynomial factors in the number of spins. We demonstrate the efficacy of polynomial functions as statistics for testing the strength of interactions in social networks in both synthetic and real world data.

We propose a dynamic edge exchangeable network model that can capture sparse connections observed in real temporal networks, in contrast to existing models which are dense. The model achieved superior link prediction accuracy on multiple data sets when compared to a dynamic variant of the blockmodel, and is able to extract interpretable time-varying community structures from the data. In addition to sparsity, the model accounts for the effect of social influence on vertices' future behaviours. Compared to the dynamic blockmodels, our model has a smaller latent space. The compact latent space requires a smaller number of parameters to be estimated in variational inference and results in a computationally friendly inference algorithm.

Knowledge transfer between tasks can improve the performance of learned models, but requires an accurate estimate of the inter-task relationships to identify the relevant knowledge to transfer. These inter-task relationships are typically estimated based on training data for each task, which is inefficient in lifelong learning settings where the goal is to learn each consecutive task rapidly from as little data as possible. To reduce this burden, we develop a lifelong learning method based on coupled dictionary learning that utilizes high-level task descriptions to model the inter-task relationships. We show that using task descriptors improves the performance of the learned task policies, providing both theoretical justification for the benefit and empirical demonstration of the improvement across a variety of learning problems. Given only the descriptor for a new task, the lifelong learner is also able to accurately predict a model for the new task through zero-shot learning using the coupled dictionary, eliminating the need to gather training data before addressing the task.

The medical research facilitates to acquire a diverse type of data from the same individual for particular cancer. Recent studies show that utilizing such diverse data results in more accurate predictions. The major challenge faced is how to utilize such diverse data sets in an effective way. In this paper, we introduce a multiple kernel based pipeline for integrative analysis of high-throughput molecular data (somatic mutation, copy number alteration, DNA methylation and mRNA) and clinical data. We apply the pipeline on Ovarian cancer data from TCGA. After multiple kernels have been generated from the weighted sum of individual kernels, it is used to stratify patients and predict clinical outcomes. We examine the survival time, vital status, and neoplasm cancer status of each subtype to verify how well they cluster. We have also examined the power of molecular and clinical data in predicting dichotomized overall survival data and to classify the tumor grade for the cancer samples. It was observed that the integration of various data types yields higher log-rank statistics value. We were also able to predict clinical status with higher accuracy as compared to using individual data types.

We will show how to perform multivariate random sampling using one of the Markov Chain Monte Carlo (MCMC) algorithms, called the Gibbs sampler. To start, what are MCMC algorithms and what are they based on? Suppose we are interested in generating a random variable with a distribution of, over . If we are not able to do this directly, we will be satisfied with generating a sequence of random variables, which in a sense tending to a distribution of . Build a Markov chain, for, whose stationary distribution is .

This chapter provides an accessible introduction for point processes, and especially Hawkes processes, for modeling discrete, inter-dependent events over continuous time. We start by reviewing the definitions and the key concepts in point processes. We then introduce the Hawkes process, its event intensity function, as well as schemes for event simulation and parameter estimation. We also describe a practical example drawn from social media data - we show how to model retweet cascades using a Hawkes self-exciting process. We presents a design of the memory kernel, and results on estimating parameters and predicting popularity. The code and sample event data are available as an online appendix

Net D-AE DBN DBM AEPerceptron RBM?GMM BayesNP SVM Supervised Supervised Unsupervised Sparse Coding SP Boosting DecisionTree Deep Neural Net RNN?Bayes Nets Modified from 16. 16 Signal Processing Information Processing Signals Processing Audio/Music Speech Image/ Animation/ Graphics Video Text/ Language Coding/ Compression Audio Coding Speech Coding Image Coding Video Coding Document Compression/ Summary Communication Voice over IP, DAB,etc 4G/5G Networks, DVB, Home Networking, etc Security Multimedia watermarking, encryption, etc. Enhancement/ Analysis De-noising/ Source separation Speech Enhancement/ Feature extraction Image/video enhancement (Clear Type), Segmentation, feature extraction Grammar checking, Text Parsing Synthesis/ Rendering Computer Music Speech Synthesis (text-to-speech) Computer Graphics/ Video Synthesis Natural Language Generation User-Interface Multi-Modal Human Computer Interaction (HCI --- Input Methods) Recognition Auditory Scene Analysis (Computer audition; e.g.