Statistical Learning
Federated Self-Supervised Learning of Multi-Sensor Representations for Embedded Intelligence
Saeed, Aaqib, Salim, Flora D., Ozcelebi, Tanir, Lukkien, Johan
Smartphones, wearables, and Internet of Things (IoT) devices produce a wealth of data that cannot be accumulated in a centralized repository for learning supervised models due to privacy, bandwidth limitations, and the prohibitive cost of annotations. Federated learning provides a compelling framework for learning models from decentralized data, but conventionally, it assumes the availability of labeled samples, whereas on-device data are generally either unlabeled or cannot be annotated readily through user interaction. To address these issues, we propose a self-supervised approach termed \textit{scalogram-signal correspondence learning} based on wavelet transform to learn useful representations from unlabeled sensor inputs, such as electroencephalography, blood volume pulse, accelerometer, and WiFi channel state information. Our auxiliary task requires a deep temporal neural network to determine if a given pair of a signal and its complementary viewpoint (i.e., a scalogram generated with a wavelet transform) align with each other or not through optimizing a contrastive objective. We extensively assess the quality of learned features with our multi-view strategy on diverse public datasets, achieving strong performance in all domains. We demonstrate the effectiveness of representations learned from an unlabeled input collection on downstream tasks with training a linear classifier over pretrained network, usefulness in low-data regime, transfer learning, and cross-validation. Our methodology achieves competitive performance with fully-supervised networks, and it outperforms pre-training with autoencoders in both central and federated contexts. Notably, it improves the generalization in a semi-supervised setting as it reduces the volume of labeled data required through leveraging self-supervised learning.
Deep Kernel Survival Analysis and Subject-Specific Survival Time Prediction Intervals
Kernel survival analysis methods predict subject-specific survival curves and times using information about which training subjects are most similar to a test subject. These most similar training subjects could serve as forecast evidence. How similar any two subjects are is given by the kernel function. In this paper, we present the first neural network framework that learns which kernel functions to use in kernel survival analysis. We also show how to use kernel functions to construct prediction intervals of survival time estimates that are statistically valid for individuals similar to a test subject. These prediction intervals can use any kernel function, such as ones learned using our neural kernel learning framework or using random survival forests. Our experiments show that our neural kernel survival estimators are competitive with a variety of existing survival analysis methods, and that our prediction intervals can help compare different methods' uncertainties, even for estimators that do not use kernels. In particular, these prediction interval widths can be used as a new performance metric for survival analysis methods.
Nonlinear ISA with Auxiliary Variables for Learning Speech Representations
Setlur, Amrith, Poczos, Barnabas, Black, Alan W
This paper extends recent work on nonlinear Independent Component Analysis (ICA) by introducing a theoretical framework for nonlinear Independent Subspace Analysis (ISA) in the presence of auxiliary variables. Observed high dimensional acoustic features like log Mel spectrograms can be considered as surface level manifestations of nonlinear transformations over individual multivariate sources of information like speaker characteristics, phonological content etc. Under assumptions of energy based models we use the theory of nonlinear ISA to propose an algorithm that learns unsupervised speech representations whose subspaces are independent and potentially highly correlated with the original non-stationary multivariate sources. We show how nonlinear ICA with auxiliary variables can be extended to a generic identifiable model for subspaces as well while also providing sufficient conditions for the identifiability of these high dimensional subspaces. Our proposed methodology is generic and can be integrated with standard unsupervised approaches to learn speech representations with subspaces that can theoretically capture independent higher order speech signals. We evaluate the gains of our algorithm when integrated with the Autoregressive Predictive Decoding (APC) model by showing empirical results on the speaker verification and phoneme recognition tasks.
Interpretabilit\'e des mod\`eles : \'etat des lieux des m\'ethodes et application \`a l'assurance
Delcaillau, Dimitri, Ly, Antoine, Vermet, Franck, Papp, Alizรฉ
Since May 2018, the General Data Protection Regulation (GDPR) has introduced new obligations to industries. By setting a legal framework, it notably imposes strong transparency on the use of personal data. Thus, people must be informed of the use of their data and must consent the usage of it. Data is the raw material of many models which today make it possible to increase the quality and performance of digital services. Transparency on the use of data also requires a good understanding of its use through different models. The use of models, even if efficient, must be accompanied by an understanding at all levels of the process that transform data (upstream and downstream of a model), thus making it possible to define the relationships between the individual's data and the choice that an algorithm could make based on the analysis of the latter. (For example, the recommendation of one product or one promotional offer or an insurance rate representative of the risk.) Models users must ensure that models do not discriminate against and that it is also possible to explain its result. The widening of the panel of predictive algorithms - made possible by the evolution of computing capacities -- leads scientists to be vigilant about the use of models and to consider new tools to better understand the decisions deduced from them . Recently, the community has been particularly active on model transparency with a marked intensification of publications over the past three years. The increasingly frequent use of more complex algorithms (\textit{deep learning}, Xgboost, etc.) presenting attractive performances is undoubtedly one of the causes of this interest. This article thus presents an inventory of methods of interpreting models and their uses in an insurance context.
Graph Gamma Process Generalized Linear Dynamical Systems
Kalantari, Rahi, Zhou, Mingyuan
We introduce graph gamma process (GGP) linear dynamical systems to model real-valued multivariate time series. For temporal pattern discovery, the latent representation under the model is used to decompose the time series into a parsimonious set of multivariate sub-sequences. In each sub-sequence, different data dimensions often share similar temporal patterns but may exhibit distinct magnitudes, and hence allowing the superposition of all sub-sequences to exhibit diverse behaviors at different data dimensions. We further generalize the proposed model by replacing the Gaussian observation layer with the negative binomial distribution to model multivariate count time series. Generated from the proposed GGP is an infinite dimensional directed sparse random graph, which is constructed by taking the logical OR operation of countably infinite binary adjacency matrices that share the same set of countably infinite nodes. Each of these adjacency matrices is associated with a weight to indicate its activation strength, and places a finite number of edges between a finite subset of nodes belonging to the same node community. We use the generated random graph, whose number of nonzero-degree nodes is finite, to define both the sparsity pattern and dimension of the latent state transition matrix of a (generalized) linear dynamical system. The activation strength of each node community relative to the overall activation strength is used to extract a multivariate sub-sequence, revealing the data pattern captured by the corresponding community. On both synthetic and real-world time series, the proposed nonparametric Bayesian dynamic models, which are initialized at random, consistently exhibit good predictive performance in comparison to a variety of baseline models, revealing interpretable latent state transition patterns and decomposing the time series into distinctly behaved sub-sequences.
Convex Representation Learning for Generalized Invariance in Semi-Inner-Product Space
Ma, Yingyi, Ganapathiraman, Vignesh, Yu, Yaoliang, Zhang, Xinhua
Invariance (defined in a general sense) has been one of the most effective priors for representation learning. Direct factorization of parametric models is feasible only for a small range of invariances, while regularization approaches, despite improved generality, lead to nonconvex optimization. In this work, we develop a convex representation learning algorithm for a variety of generalized invariances that can be modeled as semi-norms. Novel Euclidean embeddings are introduced for kernel representers in a semi-inner-product space, and approximation bounds are established. This allows invariant representations to be learned efficiently and effectively as confirmed in our experiments, along with accurate predictions.
Machine Learning May Predict Patient Satisfaction After Breast Reconstruction - Cancer Therapy Advisor
Machine learning increasingly supports physician decisions by making it easier to detect patterns in data as a means of predicting patient outcomes. In breast cancer, that now could apply to every stage of the experience, from diagnostics to mastectomy and breast reconstruction. At the annual meeting of the American Society of Clinical Oncology -- which was virtual this year, due to the ongoing coronavirus pandemic -- a consortium of researchers presented an abstract detailing how machine learning algorithms were able to correctly predict how individual patients would feel about their breast reconstruction.1 Using this tool in a clinical setting could help physicians guide patients through the recovery process in a way that better anticipates, and subsequently supports, their emotional reaction to this intensely personal medical procedure. Physician-researchers across 11 institutions in the United States and Canada trained 4 different types of machine learning algorithms -- regularized regression, Support Vector Machine, Neural Network, Regression Tree -- to predict with 95% accuracy whether a specific patient would be satisfied or dissatisfied with their breast reconstruction 2 years after their operation.
Predicting COVID-19 With Machine Learning
Predicting COVID-19 in India using Machine Learning.In this session, we will take a COVID-19 dataset and understand how the disease has spread across different states in India. We will perform some data manipulation and data visualization operations on top of the dataset. Great Learning brings you this live session on'Predicting COVID-19 in India using Machine Learning'.In this session, we will take a COVID-19 dataset and understand how the disease has spread across different states in India. We will perform some data manipulation and data visualization operations on top of the dataset.
Top 10 Best FREE Artificial Intelligence Courses
Most of the Machine Learning, Deep Learning, Computer Vision, NLP job positions, or in general every Artificial Intelligence (AI) job position requires you to have at least a bachelor's degree in Computer Science, Electrical Engineering, or some similar field. If your degree comes from some of the world's best universities than your chances might be higher in beating the competition on your job interview. But looking realistically, not most of the people can afford to go to the top universities in the world simply because not most of us are geniuses and don't have thousands of dollars, or come from some poor country (like we do). No with the high demand of skilled professionals from these fields, there are exceptions being made, so we can see that people who don't come from these fields, are learning and adjusting themselves in order to get that paycheck. In this article, we are going to list some of the free Artificial Intelligence courses that come from Harvard University, MIT University, and Stanford University that anyone can attend, no matter where they live.