Fast matrix algorithms have become the fundamental tools of machine learning in big data era. The generalized matrix regression problem is widely used in the matrix approximation such as CUR decomposition, kernel matrix approximation, and stream singular value decomposition (SVD), etc. In this paper, we propose a fast generalized matrix regression algorithm (Fast GMR) which utilizes sketching technique to solve the GMR problem efficiently. Given error parameter $0<\epsilon<1$, the Fast GMR algorithm can achieve a $(1+\epsilon)$ relative error with the sketching sizes being of order $\cO(\epsilon^{-1/2})$ for a large group of GMR problems. We apply the Fast GMR algorithm to the symmetric positive definite matrix approximation and single pass singular value decomposition and they achieve a better performance than conventional algorithms. Our empirical study also validates the effectiveness and efficiency of our proposed algorithms.

One critical challenge for applying today's Artificial Intelligence (AI) technologies to real-world applications is the common existence of data silos across different organizations. Due to legal, privacy and other practical constraints, data from different organizations cannot be easily integrated. Federated learning (FL), especially the vertical FL (VFL), allows multiple parties having different sets of attributes about the same user collaboratively build models while preserving user privacy. However, communication overhead is a principal bottleneck since the existing VFL protocols require per-iteration communications among all parties. In this paper, we propose the Federated Stochastic Block Coordinate Descent (FedBCD) to effectively reduce the communication rounds for VFL. We show that when the batch size, sample size, and the local iterations are selected appropriately, the algorithm requires $\mathcal{O}(\sqrt{T})$ communication rounds to achieve $\mathcal{O}(1/\sqrt{T})$ accuracy. Finally, we demonstrate the performance of FedBCD on several models and datasets, and on a large-scale industrial platform for VFL.

Recent research on network embedding in hyperbolic space have proven successful in several applications. However, nodes in real world networks tend to interact through several distinct channels. Simple aggregation or ignorance of this multiplexity will lead to misleading results. On the other hand, there exists redundant information between different interaction patterns between nodes. Recent research reveals the analogy between the community structure and the hyperbolic coordinate. To learn each node's effective embedding representation while reducing the redundancy of multiplex network, we then propose a unified framework combing multiplex network hyperbolic embedding and multiplex community detection. The intuitive rationale is that high order node embedding approach is expected to alleviate the observed network's sparse and noisy structure which will benefit the community detection task. On the contrary, the improved community structure will also guide the node embedding task. To incorporate the common features between channels while preserving unique features, a random walk approach which traversing in latent multiplex hyperbolic space is proposed to detect the community across channels and bridge the connection between node embedding and community detection. The proposed framework is evaluated on several network tasks using different real world dataset. The results demonstrates that our framework is effective and efficiency compared with state-of-the-art approaches.

This paper introduces a new approach to learning from i.i.d. data under model miss-specification. This approach casts the problem of learning as minimizing the expected code-length of a Bayesian mixture code. To solve this problem, we build on PAC-Bayes bounds, information theory and a new family of second-order Jensen bounds. The key insight of this paper is that the use of the standard (first-order) Jensen bounds in learning is suboptimal when our model class is miss-specified (i.e. it does not contain the data generating distribution). As a consequence of this insight, this work provides strong theoretical arguments explaining why the Bayesian posterior is not optimal for making predictions that generalize under model miss-specification because the Bayesian posterior is directly related to the use of first-order Jensen bounds. We then argue for the use of second-order Jensen bounds, which leads to new families of learning algorithms. In this work, we introduce novel variational and ensemble learning methods based on the minimization of a novel family of second-order PAC-Bayes bounds over the expected code-length of a Bayesian mixture code. Using this new framework, we also provide novel hypotheses of why parameters in a flat minimum generalize better than parameters in a sharp minimum.

Leonel Garcia-Marques CICPSI Faculdade de Psicologia Universidade de Lisboa Portugal garcia_marques@sapo.pt Abstract --An agent, artificial or human, must be continuously adjusting its behaviour in order to thrive in a more or less demanding environment. An artificial agent with the ability to predict human emotional valence in a geospatial and temporal context requires proper adaptation to its mobile device environment with resource consumption strict restrictions (e.g., power from battery). The developed distributed system includes a mobile device embodied agent ( SensAI) plus Cloud-expanded ( Expanse) cognition and memory resources. The system is designed with several adaptive mechanisms in a best effort for the agent to cope with its interacting humans and to be resilient on collecting data for machine learning towards prediction. These mechanisms encompass homeostatic-like adjustments such as auto recovering from an unexpected failure in the mobile device, forgetting repeated data to save local memory, adjusting actions to a proper moment (e.g., notify only when human is interacting), and the Expanse complementary learning algorithms' parameters with auto adjustments. Regarding emotional valence prediction performance, results from a comparison study between state-of-the-art algorithms revealed Extreme Gradient Boosting on average the best model for prediction with efficient energy use, and explainable using feature importance inspection. Therefore, this work contributes with a smartphone sensing-based system, distributed in the Cloud, robust to unexpected behaviours from humans and the environment, able to predict emotional valence states with very good performance. I NTRODUCTION The scientific evidence of epigenetics reveal on/off mechanisms inside chromosomes of human agents and reinforces the importance of any entity continuous adaptation to its environment.

A major component of overfitting in model-free reinforcement learning (RL) involves the case where the agent may mistakenly correlate reward with certain spurious features from the observations generated by the Markov Decision Process (MDP). We provide a general framework for analyzing this scenario, which we use to design multiple synthetic benchmarks from only modifying the observation space of an MDP. When an agent overfits to different observation spaces even if the underlying MDP dynamics is fixed, we term this observational overfitting. Our experiments expose intriguing properties especially with regards to implicit regularization, and also corroborate results from previous works in RL generalization and supervised learning (SL).

This course is about artificial neural networks. Artificial intelligence and machine learning are getting more and more popular nowadays. In the beginning, other techniques such as Support Vector Machines outperformed neural networks, but in the 21th century neural networks again gain popularity. In spite of the slow training procedure, neural networks can be very powerful. In the first part of the course you will learn about the theoretical background of neural networks, later you will learn how to implement them.

Machine learning models are fantastic; they can recognize objects in videos; they can automatically generate captions for images and accurately classify pictures of cats and dogs (sometimes). This article will provide a surface level understanding of what happens underneath the hood of Machine learning models. More specifically, we will be exploring the'backbone algorithms' that enable these machine learning models to learn. The'backbone algorithms' are called Optimization algorithms. Below are some definitions of keywords you will encounter within this article, and optimization algorithm is amongst the provided descriptions.

Provide an overview of Azure Machine Learning Studio and high level architecture.

In this Arduno Machine learning project we're going to use an accelerometer sensor to identify the gestures you play. This is a remake of the project found on the Tensorflow blog. We're going to use a lot less powerful chip in this tutorial, tough: an Arduino Nano (old generation), equipped with 32 kb of flash and only 2 kb of RAM. We're going to use the accelerations along the 3 axis (X, Y, Z) coming from an IMU to infer which gesture we're playing. We'll use a fixed number of recordings (NUM_SAMPLES) starting from the first detection of movement.