We propose a spectral clustering algorithm for analyzing the dependence structure of multivariate extremes. More specifically, we focus on the asymptotic dependence of multivariate extremes characterized by the angular or spectral measure in extreme value theory. Our work studies the theoretical performance of spectral clustering based on a random $k$-nearest neighbor graph constructed from an extremal sample, i.e., the angular part of random vectors for which the radius exceeds a large threshold. In particular, we derive the asymptotic distribution of extremes arising from a linear factor model and prove that, under certain conditions, spectral clustering can consistently identify the clusters of extremes arising in this model. Leveraging this result we propose a simple consistent estimation strategy for learning the angular measure. Our theoretical findings are complemented with numerical experiments illustrating the finite sample performance of our methods.

In contrastive representation learning, data representation is trained so that it can classify the image instances even when the images are altered by augmentations. However, depending on the datasets, some augmentations can damage the information of the images beyond recognition, and such augmentations can result in collapsed representations. We present a partial solution to this problem by formalizing a stochastic encoding process in which there exist a tug-of-war between the data corruption introduced by the augmentations and the information preserved by the encoder. We show that, with the infoMax objective based on this framework, we can learn a data-dependent distribution of augmentations to avoid the collapse of the representation.

Taxicab service plays an essential and irreplaceable role in urban traffic system [Ji et al., 2020]. For example, in New York City, there are more than 21,000 taxi drivers and more than 80,000 ride-sharing drivers. Compared to other means of daily transportation, such as bus and subway, taxis usually offers a better trip experience in terms of comfort, convenience, and travel time accommodation. Thus, it has been a long-standing central issue to improve the efficiency of vehicle mobility by optimizing the route recommendation for drivers for taxi services in big cities like New York, Tokyo, and Beijing [Yuan et al., 2011, Zheng et al., 2014]. Based on large-scale taxi trace data, there is an extensive literature on route recommendation systems. Some studies focus on the traditional optimization method. For example, Qu et al. [2014] proposed a cost-efficient objective function and developed a greedy method to maximize the potential net profit. Similar methods can be found in [Ding et al., 2013, Zhou et al., 2016]. Stochastic optimization methods (e.g., simulated annealing -SA-) and parallel computing techniques have also been applied to route recommendation problems to speed up the route searching tasks (see [Ye This manuscript has been accepted by ACM Transactions on Intelligent Systems and Technology on April 25, 2021.

Protein complex formation is a central problem in biology, being involved in most of the cell's processes, and essential for applications, e.g. We tackle rigid body protein-protein docking, i.e., computationally predicting the 3D structure of a protein-protein complex from the individual unbound structures, assuming no conformational change within the proteins happens during binding. We design a novel pairwise-independent SE(3)-equivariant graph matching network to predict the rotation and translation to place one of the proteins at the right docked position relative to the second protein. We mathematically guarantee a basic principle: the predicted complex is always identical regardless of the initial locations and orientations of the two structures. Empirically, we achieve significant running time improvements and often outperform existing docking software despite not relying on heavy candidate sampling, structure refinement, or templates. Besides their complex three-dimensional nature, Figure 1: Different views of the 3D structure proteins dynamically alter their function and structure of a protein complex. In particular, protein interactions are involved in various biological processes including signal transduction, protein synthesis, DNA replication and repair. Molecular docking is key to understanding protein interactions' mechanisms and effects, and, subsequently, to developing therapeutic interventions.

as in the previous article, I have given you an introduction to polynomial regression now I will tell you how to make a basic polynomial regression model in this article with some lines of codes. the…

Application of machine learning in earth sciences is the use of computer systems to classify, cluster, identify and analyze vast and complex data in earth science study, for example, geological mapping, gas leakage detection and geological features identification. Machine learning (ML) is a type of Artificial Intelligence (AI) that allows computer systems to interpret data while eliminating the need for explicit instructions and programming. The Earth system can be subdivided into four major components including the solid earth, atmosphere, hydrosphere and biosphere[3]. A variety of algorithms may be applied depending on the nature of the earth science exploration. Some algorithms may perform significantly better than others for particular objectives. For example, Convolutional Neural Networks (CNN) are good at interpreting images, Artificial Neural Network (ANN) performs well in soil classification[4] but more computationally expensive to train than Support Vector Machine (SVM) learning.

Hi everyone, I'm reposting all of my old blogs, as my account got hacked. This blog was originally published on March 6, 2019. Within the field of machine learning, there are two main types of tasks: supervised, and unsupervised. The main difference between the two types is that supervised learning is done using a ground truth, or in other words, we have prior knowledge of what the output values for our samples should be. Therefore, the goal of supervised learning is to learn a function that, given a sample of data and desired outputs, best approximates the relationship between input and output observable in the data.

Community detection for un-weighted networks has been widely studied in network analysis, but the case of weighted networks remains a challenge. In this paper, a Distribution-Free Models (DFM) is proposed for networks in which nodes are partitioned into different communities. DFM is a general, interpretable and identifiable model for both un-weighted networks and weighted networks. The proposed model does not require prior knowledge on a specific distribution for elements of adjacency matrix but only the expected value. The distribution-free property of DFM even allows adjacency matrix to have negative elements. We develop an efficient spectral algorithm to fit DFM. By introducing a noise matrix, we build a theoretic framework on perturbation analysis to show that the proposed algorithm stably yields consistent community detection under DFM. Numerical experiments on both synthetic networks and two social networks from literature are used to illustrate the algorithm.

In this paper we study the statistical properties of Principal Components Regression with Laplacian Eigenmaps (PCR-LE), a method for nonparametric regression based on Laplacian Eigenmaps (LE). PCR-LE works by projecting a vector of observed responses ${\bf Y} = (Y_1,\ldots,Y_n)$ onto a subspace spanned by certain eigenvectors of a neighborhood graph Laplacian. We show that PCR-LE achieves minimax rates of convergence for random design regression over Sobolev spaces. Under sufficient smoothness conditions on the design density $p$, PCR-LE achieves the optimal rates for both estimation (where the optimal rate in squared $L^2$ norm is known to be $n^{-2s/(2s + d)}$) and goodness-of-fit testing ($n^{-4s/(4s + d)}$). We also show that PCR-LE is \emph{manifold adaptive}: that is, we consider the situation where the design is supported on a manifold of small intrinsic dimension $m$, and give upper bounds establishing that PCR-LE achieves the faster minimax estimation ($n^{-2s/(2s + m)}$) and testing ($n^{-4s/(4s + m)}$) rates of convergence. Interestingly, these rates are almost always much faster than the known rates of convergence of graph Laplacian eigenvectors to their population-level limits; in other words, for this problem regression with estimated features appears to be much easier, statistically speaking, than estimating the features itself. We support these theoretical results with empirical evidence.

We present a novel method for reducing the computational complexity of rigorously estimating the partition functions (normalizing constants) of Gibbs (Boltzmann) distributions, which arise ubiquitously in probabilistic graphical models. A major obstacle to practical applications of Gibbs distributions is the need to estimate their partition functions. The state of the art in addressing this problem is multi-stage algorithms, which consist of a cooling schedule, and a mean estimator in each step of the schedule. While the cooling schedule in these algorithms is adaptive, the mean estimation computations use MCMC as a black-box to draw approximate samples. We develop a doubly adaptive approach, combining the adaptive cooling schedule with an adaptive MCMC mean estimator, whose number of Markov chain steps adapts dynamically to the underlying chain. Through rigorous theoretical analysis, we prove that our method outperforms the state of the art algorithms in several factors: (1) The computational complexity of our method is smaller; (2) Our method is less sensitive to loose bounds on mixing times, an inherent component in these algorithms; and (3) The improvement obtained by our method is particularly significant in the most challenging regime of high-precision estimation. We demonstrate the advantage of our method in experiments run on classic factor graphs, such as voting models and Ising models.