The Incremental K-means (IKM), an improved version of K-means (KM), was introduced to improve the clustering quality of KM significantly. However, the speed of IKM is slower than KM. My thesis proposes two algorithms to speed up IKM while remaining the quality of its clustering result approximately. The first algorithm, called Divisive K-means, improves the speed of IKM by speeding up its splitting process of clusters. Testing with UCI Machine Learning data sets, the new algorithm achieves the empirically global optimum as IKM and has lower complexity, $O(k*log_{2}k*n)$, than IKM, $O(k^{2}n)$. The second algorithm, called Parallel Two-Phase K-means (Par2PK-means), parallelizes IKM by employing the model of Two-Phase K-means. Testing with large data sets, this algorithm attains a good speedup ratio, closing to the linearly speed-up ratio.

Canonical correlation analysis (CCA) is a classical and important multivariate technique for exploring the relationship between two sets of continuous variables. CCA has applications in many fields, such as genomics and neuroimaging. It can extract meaningful features as well as use these features for subsequent analysis. Although some sparse CCA methods have been developed to deal with high-dimensional problems, they are designed specifically for continuous data and do not consider the integer-valued data from next-generation sequencing platforms that exhibit very low counts for some important features. We propose a model-based probabilistic approach for correlation and canonical correlation estimation for two sparse count data sets (PSCCA). PSCCA demonstrates that correlations and canonical correlations estimated at the natural parameter level are more appropriate than traditional estimation methods applied to the raw data. We demonstrate through simulation studies that PSCCA outperforms other standard correlation approaches and sparse CCA approaches in estimating the true correlations and canonical correlations at the natural parameter level. We further apply the PSCCA method to study the association of miRNA and mRNA expression data sets from a squamous cell lung cancer study, finding that PSCCA can uncover a large number of strongly correlated pairs than standard correlation and other sparse CCA approaches.

Graph clustering is widely used in analysis of biological networks, social networks and etc. For over a decade many graph clustering algorithms have been published, however a comprehensive and consistent performance comparison is not available. In this paper we benchmarked more than 70 graph clustering programs to evaluate their runtime and quality performance for both weighted and unweighted graphs. We also analyzed the characteristics of ground truth that affects the performance. Our work is capable to not only supply a start point for engineers to select clustering algorithms but also could provide a viewpoint for researchers to design new algorithms.

There are two famous function decomposition methods in math: 1) Taylor Series and 2) Fourier Series. The Fourier series developed into the Fourier spectrum, which was applied to signal analysis. However, Because a function without a functional expression cannot be solved for its Taylor series, Taylor Series has rarely been used in engineering. Here we have solved this problem, learned from Fourier, developed Taylor series, constructed a relation spectrum, and applied it to system analysis. Specific engineering application: the knowledge of the intuitive link between muscle activity and the finger movement is vital for the design of commercial prosthetic hands that do not need user pre-training. However, this link has yet to be understood due to the complexity of human hand. In this study, the relation spectrum was developed for the first time and applied to analyze the muscle-finger system. We established controllable and human-readable polynomial neural network (CR-PNN) models for six degrees of freedom ( DOFs) in 8 subjects. Multiple fingers may be controlled by a single muscle, or multiple muscles may control a single finger. Thus, the research is based on two aspects: muscle synergy and muscle coupling for hand. The research gave the relation spectrum of the muscle-finger system and the knowledge of muscle coupling. The article is very short but significant. The contributions of this paper can be divided into two parts: (1) The findings of hand can contribute to design prosthetic hands. (2) The relation spectrum using CR-PNN can provide a reference for analyzing complex systems in multiple areas. (We're strong believers in Open Source, and provide CR-PNN code for others. GitHub: https://github.com/liugang1234567/CR-PNN#cr-pnn. )

A methodology to perform topological regularization via information filtering network is introduced. This methodology can be directly applied to sparse modeling with the vast family of elliptical probability distributions. It can also be directly implemented for $L_0$ norm regularized multicollinear regression. In this paper, I describe in detail an application to sparse modeling with multivariate Student-t. A specific $L_0$ norm regularized expectation-maximization likelihood maximization procedure is proposed for this sparse Student-t case. Examples with real data from stock prices log-returns and from artificially generated data demonstrate applicability, performances, and potentials of this methodology.

Neural networks have seen limited use in prediction for high-dimensional data with small sample sizes, because they tend to overfit and require tuning many more hyperparameters than existing off-the-shelf machine learning methods. With small modifications to the network architecture and training procedure, we show that dense neural networks can be a practical data analysis tool in these settings. The proposed method, Ensemble by Averaging Sparse-Input Hierarchical networks (EASIER-net), appropriately prunes the network structure by tuning only two L1-penalty parameters, one that controls the input sparsity and another that controls the number of hidden layers and nodes. The method selects variables from the true support if the irrelevant covariates are only weakly correlated with the response; otherwise, it exhibits a grouping effect, where strongly correlated covariates are selected at similar rates. On a collection of real-world datasets with different sizes, EASIER-net selected network architectures in a data-adaptive manner and achieved higher prediction accuracy than off-the-shelf methods on average.

It is difficult to decide upon the efficacy of an online sale simply from the discount offered on commodities. Different features have different influence on the price of a product which must be taken into consideration when determining the significance of a discount. In this paper we have proposed a machine learning based heuristic to quantify the \textit{"significance"} of the discount offered on any commodity. Our proposed technique can quantify the significance of the discount based on features and the original price, and hence can guide a buyer during a sale season by predicting the efficacy of the sale. We have applied this technique on the Flipkart Summer Sale dataset using Support Vector Machine, which predicts the efficacy of the sale with an accuracy of 91.11\%. Our result shows that very few mobile phones have a significant discount during the Flipkart Summer Sale.

While working on different Machine Learning techniques for Data Analysis, we deal with hundreds or thousands of variables. Most of the variables are correlated with each other. Principal Component Analysis and Factor Analysis techniques are used to deal with such scenarios. Principal Component Analysis (PCA) is an unsupervised statistical technique algorithm. PCA is a "dimensionality reduction" method.

The application of federated extreme gradient boosting to mobile crowdsensing apps brings several benefits, in particular high performance on efficiency and classification. However, it also brings a new challenge for data and model privacy protection. Besides it being vulnerable to Generative Adversarial Network (GAN) based user data reconstruction attack, there is not the existing architecture that considers how to preserve model privacy. In this paper, we propose a secret sharing based federated learning architecture FedXGB to achieve the privacy-preserving extreme gradient boosting for mobile crowdsensing. Specifically, we first build a secure classification and regression tree (CART) of XGBoost using secret sharing. Then, we propose a secure prediction protocol to protect the model privacy of XGBoost in mobile crowdsensing. We conduct a comprehensive theoretical analysis and extensive experiments to evaluate the security, effectiveness, and efficiency of FedXGB. The results indicate that FedXGB is secure against the honest-but-curious adversaries and attains less than 1% accuracy loss compared with the original XGBoost model.

Water demand is a highly important variable for operational control and decision making. Hence, the development of accurate forecasts is a valuable field of research to further improve the efficiency of water utilities. Focusing on probabilistic multi-step-ahead forecasting, a time series model is introduced, to capture typical autoregressive, calendar and seasonal effects, to account for time-varying variance, and to quantify the uncertainty and path-dependency of the water demand process. To deal with the high complexity of the water demand process a high-dimensional feature space is applied, which is efficiently tuned by an automatic shrinkage and selection operator (lasso). It allows to obtain an accurate, simple interpretable and fast computable forecasting model, which is well suited for real-time applications. The complete probabilistic forecasting framework allows not only for simulating the mean and the marginal properties, but also the correlation structure between hours within the forecasting horizon. For practitioners, complete probabilistic multi-step-ahead forecasts are of considerable relevance as they provide additional information about the expected aggregated or cumulative water demand, so that a statement can be made about the probability with which a water storage capacity can guarantee the supply over a certain period of time. This information allows to better control storage capacities and to better ensure the smooth operation of pumps. To appropriately evaluate the forecasting performance of the considered models, the energy score (ES) as a strictly proper multidimensional evaluation criterion, is introduced. The methodology is applied to the hourly water demand data of a German water supplier.