Neuronal circuits formed in the brain are complex with intricate connection patterns. Such a complexity is also observed in the retina as a relatively simple neuronal circuit. A retinal ganglion cell receives excitatory inputs from neurons in previous layers as driving forces to fire spikes. Analytical methods are required that can decipher these components in a systematic manner. Recently a method termed spike-triggered non-negative matrix factorization (STNMF) has been proposed for this purpose. In this study, we extend the scope of the STNMF method. By using the retinal ganglion cell as a model system, we show that STNMF can detect various biophysical properties of upstream bipolar cells, including spatial receptive fields, temporal filters, and transfer nonlinearity. In addition, we recover synaptic connection strengths from the weight matrix of STNMF. Furthermore, we show that STNMF can separate spikes of a ganglion cell into a few subsets of spikes where each subset is contributed by one presynaptic bipolar cell. Taken together, these results corroborate that STNMF is a useful method for deciphering the structure of neuronal circuits.

The acceleration technique first introduced by Nesterov for gradient descent is widely used in many machine learning applications, however it is not yet well-understood. Recently, significant progress has been made to close this understanding gap by using a continuous-time dynamical system perspective associated with gradient-based methods. In this paper, we extend this perspective by considering the continuous limit of the family of relaxed Alternating Direction Method of Multipliers (ADMM). We also introduce two new families of relaxed and accelerated ADMM algorithms, one follows Nesterov's acceleration approach and the other is inspired by Polyak's Heavy Ball method, and derive the continuous limit of these families of relaxed and accelerated algorithms as differential equations. Then, using a Lyapunov stability analysis of the dynamical systems, we obtain rate-of-convergence results for convex and strongly convex objective functions.

It is a practical, hands-on course, i.e. we will spend some time dealing with some of the theoretical concepts related to both statistical and machine learning regression analysis. However, majority of the course will focus on implementing different techniques on real data and interpret the results. After each video you will learn a new concept or technique which you may apply to your own projects.

We propose to use deep neural networks for generating samples in Monte Carlo integration. Our work is based on non-linear independent component analysis, which we extend in numerous ways to improve performance and enable its application to integration problems. First, we introduce piecewise-polynomial coupling transforms that greatly increase the modeling power of individual coupling layers. Second, we propose to preprocess the inputs of neural networks using one-blob encoding, which stimulates localization of computation and improves inference. Third, we derive a gradient-descent-based optimization for the KL and the $\chi^2$ divergence for the specific application of Monte Carlo integration with stochastic estimates of the target distribution. Our approach enables fast and accurate inference and efficient sample generation independent of the dimensionality of the integration domain. We demonstrate the benefits of our approach for generating natural images and in two applications to light-transport simulation. First, we show how to learn joint path-sampling densities in primary sample space and how to importance sample multi-dimensional path prefixes thereof. Second, we use our technique to extract conditional directional densities driven by the triple product of the rendering equation, and leverage them for path guiding. In all applications, our approach yields on-par or higher performance at equal sample count than competing techniques.

In the traditional framework of spectral learning of stochastic time series models, model parameters are estimated based on trajectories of fully recorded observations. However, real-world time series data often contain missing values, and worse, the distributions of missingness events over time are often not independent of the visible process. Recently, a spectral OOM learning algorithm for time series with missing data was introduced and proved to be consistent, albeit under quite strong conditions. Here we refine the algorithm and prove that the original strong conditions can be very much relaxed. We validate our theoretical findings by numerical experiments, showing that the algorithm can consistently handle missingness patterns whose dynamic interacts with the visible process.

A capsule is a collection of neurons which represents different variants of a pattern in the network. The routing scheme ensures only certain capsules which resemble lower counterparts in the higher layer should be activated. However, the computational complexity becomes a bottleneck for scaling up to larger networks, as lower capsules need to correspond to each and every higher capsule. To resolve this limitation, we approximate the routing process with two branches: a master branch which collects primary information from its direct contact in the lower layer and an aide branch that replenishes master based on pattern variants encoded in other lower capsules. Compared with previous iterative and unsupervised routing scheme, these two branches are communicated in a fast, supervised and one-time pass fashion. The complexity and runtime of the model are therefore decreased by a large margin. Motivated by the routing to make higher capsule have agreement with lower capsule, we extend the mechanism as a compensation for the rapid loss of information in nearby layers. We devise a feedback agreement unit to send back higher capsules as feedback. It could be regarded as an additional regularization to the network. The feedback agreement is achieved by comparing the optimal transport divergence between two distributions (lower and higher capsules). Such an add-on witnesses a unanimous gain in both capsule and vanilla networks. Our proposed EncapNet performs favorably better against previous state-of-the-arts on CIFAR10/100, SVHN and a subset of ImageNet.

This thesis employs statistical learning technique to analyze, predict and solve the fixed charge network flow (FCNF) problem, which is common encountered in many real-world network problems. The cost structure for flows in the FCNF involves both fixed and variable costs. The FCNF problem is modeled mixed binary linear programs and can be solved with standard commercial solvers, which use branch and bound algorithm. This problem is important for its widely applications and solving challenges. There does not exist a efficient algorithm to solve this problem optimally due to lacking tight bounds. To the best of our knowledge, this is the first work that employs statistical learning technique to analyze the optimal flow of the FCNF problem. Most algorithms developed to solve the FCNF problem are based on the cost structure, relaxation, etc. We start from the network characteristics and explore the relationship between properties of nodes, arcs and networks and the optimal flow. This is a bi-direction approach and the findings can be used to locate the features that affect the optimal flow most significantly, predict the optimal arcs and provide information to solve the FCNF problem. In particular, we define 33 features based on the network characteristics, from which using step wise regression, we identify 26 statistical significant predictors for logistic regression to predict which arcs will have positive flow in the optimal solutions. The predictive model achieves 88% accuracy and the area under receiver operating characteristic curve is 0.95. Two applications are investigated. Firstly, the predictive results can be used directly as component critical index. The failure of arcs with higher critical index result in more cost increase over the entire network.

XGBoost (or eXteme Gradient Boosting) is not to be introduced anymore, proved relevant in only too many data science competitions, is still one model that is tricky to fine-tune if you have only been starting playing with it. Because if you have big datasets, and you run a naive grid search on 5 different parameters and having for each of them 5 possible values, then you'll have 5⁵ 3,125 iterations to go. If one iteration takes 10 minutes to run, you'll have more than 21 days to wait before getting your parameters (I don't talk about Python crashing, without letting you know, and you waiting too long before realizing it). I suppose here that you made correctly your job of feature engineering first. Specifically with categorical features, since XGBoost does not take categorical features in input.

These days having skills in deep learning and machine learning is one of the most trending things in the tech world right now, and businesses are looking to hire developers who possess good knowledge in machine learning. In fact, Java has become a usual norm for implementing new machine learning algorithms these days. There are so many benefits of learning Java and is accepted by the people in machine learning community, easy maintenance, marketability, and readability, among others. If you want to integrate machine learning into your existing Java business applications then you must hire Java developers for the same. In this post, we will list down some of the best libraries for implementing machine learning in existing Java applications.

This work addresses the problem of author name homonymy in the Web of Science. Aiming for an efficient, simple and straightforward solution, we introduce a novel probabilistic similarity measure for author name disambiguation based on feature overlap. Using the researcher-ID available for a subset of the Web of Science, we evaluate the application of this measure in the context of agglomeratively clustering author mentions. We focus on a concise evaluation that shows clearly for which problem setups and at which time during the clustering process our approach works best. In contrast to most other works in this field, we are skeptical towards the performance of author name disambiguation methods in general and compare our approach to the trivial single-cluster baseline. Our results are presented separately for each correct clustering size as we can explain that, when treating all cases together, the trivial baseline and more sophisticated approaches are hardly distinguishable in terms of evaluation results. Our model shows state-of-the-art performance for all correct clustering sizes without any discriminative training and with tuning only one convergence parameter.