ABSTRACT Motivated by the ever-increasing demands for limited communication bandwidth and low-power consumption, we propose a new methodology, named joint V ariational Autoen-coders with Bernoulli mixture models (V AB), for performing clustering in the compressed data domain. The idea is to reduce the data dimension by V ariational Autoencoders (V AEs) and group data representations by Bernoulli mixture models (BMMs). Once jointly trained for compression and clustering, the model can be decomposed into two parts: a data vendor that encodes the raw data into compressed data, and a data consumer that classifies the received (compressed) data. To enable training using the gradient descent algorithm, we propose to use the Gumbel-Softmax distribution to resolve the infeasibility of the back-propagation algorithm when assessing categorical samples. Index T erms -- Clustering, V ariational Autoencoder (V AE), Bernoulli Mixture Model (BMM) 1. INTRODUCTION Clustering is a fundamental task with applications in medical imaging, social network analysis, bioinformatics, computer graphics, etc. Applying classical clustering methods directly to high dimensional data may be computational inefficient and suffer from instability.

This paper focuses on projection-free methods for solving smooth Online Convex Optimization (OCO) problems. Existing projection-free methods either achieve suboptimal reg ret bounds or have high per-iteration computational costs. To fi ll this gap, two efficient projection-free online methods call ed ORGFW and MORGFW are proposed for solving stochastic and adversarial OCO problems, respectively. By employing a recursive gradient estimator, our methods achieve optimal regret bounds (up to a logarithmic factor) while possessing low per-iteration computational costs. Experimen tal results demonstrate the efficiency of the proposed methods compared to state-of-the-arts.

In this era of big data, feature selection techniques, which have long been proven to simplify the model, makes the model more comprehensible, speed up the process of learning, have become more and more important. Among many developed methods, forward, backward and stepwise feature selection regression remained widely used due to their simplicity and efficiency. However, they are not sufficient enough when it comes to large datasets. In this paper, we analyze the issues associated with those approaches and introduce a novel algorithm that may boost the speed up to 65.77% compared to stepwise while maintaining good performance compared to stepwise selection in terms of the number of selected features and error rates.

ABSTRACT In this paper, we derive generic bounds on the maximum deviations in prediction errors for sequential prediction vi a an information-theoretic approach. The fundamental bounds a re shown to depend only on the conditional entropy of the data point to be predicted given the previous data points. In the asymptotic case, the bounds are achieved if and only if the prediction error is white and uniformly distributed. Index T erms -- Information-theoretic learning, sequential learning, sequential prediction, bounds on performan ce, sequence prediction 1. INTRODUCTION Nowadays machine learning techniques are becoming more and more prevalent in real-time systems such as real-time si g-nal processing, feedback control, and robotics systems. In such systems, on one hand, decisions on the actions are to be made in a sequential manner (sequential decision making); on the other hand, dynamics of the systems as well as the environment that are determined by physical laws will play an indispensable role and must be taken into consideration (interaction with real world).

The covariance structure of multivariate functional data can be highly complex, especially if the multivariate dimension is large, making extension of statistical methods for standard multivariate data to the functional data setting quite challenging. For example, Gaussian graphical models have recently been extended to the setting of multivariate functional data by applying multivariate methods to the coefficients of truncated basis expansions. However, a key difficulty compared to multivariate data is that the covariance operator is compact, and thus not invertible. The methodology in this paper addresses the general problem of covariance modeling for multivariate functional data, and functional Gaussian graphical models in particular. As a first step, a new notion of separability for multivariate functional data is proposed, termed partial separability, leading to a novel Karhunen-Lo\`eve-type expansion for such data. Next, the partial separability structure is shown to be particularly useful in order to provide a well-defined Gaussian graphical model that can be identified with a sequence of finite-dimensional graphical models, each of fixed dimension. This motivates a simple and efficient estimation procedure through application of the joint graphical lasso. Empirical performance of the method for graphical model estimation is assessed through simulation and analysis of functional brain connectivity during a motor task.

Auto-Model: Utilizing Research Papers and HPO Techniques to Deal with the CASH problem Chunnan Wang, Hongzhi Wang, Tianyu Mu, Jianzhong Li, Hong Gao Department of Computer Science Harbin Institute of T echnology Harbin, China {WangChunnan, wangzh, mutianyu, lijzh, honggao }@hit.edu.cn Abstract --In many fields, a mass of algorithms with completely different hyperparameters have been developed to address the same type of problems. Choosing the algorithm and hyperpa-rameter setting correctly can promote the overall performance greatly, but users often fail to do so due to the absence of knowledge. How to help users to effectively and quickly select the suitable algorithm and hyperparameter settings for the given task instance is an important research topic nowadays, which is known as the CASH problem. In this paper, we design the Auto-Model approach, which makes full use of known information in the related research paper and introduces hyperparameter optimization techniques, to solve the CASH problem effectively. Auto-Model tremendously reduces the cost of algorithm implementations and hyperparameter configuration space, and thus capable of dealing with the CASH problem efficiently and easily. T o demonstrate the benefit of Auto-Model, we compare it with classical Auto-Weka approach. The experimental results show that our proposed approach can provide superior results and achieves better performance in a short time. Index T erms--Algorithm selection, Hyperparameter optimization, Combined algorithm selection and hyperparameter optimization problem, Auto-Weka, Classification algorithms I. I NTRODUCTION In many fields, such as machine learning, data mining, artificial intelligence and constraint satisfaction, a variety of algorithms and heuristics have been developed to address the same type of problem [1], [2]. Each of these algorithms has its own advantages and disadvantages, and often they are complementary in the sense that one algorithm works well when others fail and vice versa [2]. If we are capable of selecting the algorithm and hyperparameter setting best suited to the task instance, any particular task instance will be well solved, and our ability of dealing with the problem will be improved considerably [3]. However, it is not trivial to achieve this goal. There are a mass of powerful and different algorithms to deal with a certain problem, and these algorithms have completely different hyperparameters, which have great effect on their performance. Even domain experts cannot easily and correctly select the appropriate algorithm with corresponding optimal hyperparameters from such a huge and complex choice space.

Autoencoders enable data dimensionality reduction and are a key component of many (deep) learning systems. This article explores the use of the XCSF online evolutionary reinforcement learning system to perform autoencoding. Initial results using a neural network representation and combining artificial evolution with stochastic gradient descent, suggest it is an effective approach to data reduction. The approach adaptively subdivides the input domain into local approximations that are simpler than a global neural network solution. By allowing the number of neurons in the autoencoders to evolve, this further enables the emergence of an ensemble of structurally heterogeneous solutions to cover the problem space. In this case, networks of differing complexity are typically seen to cover different areas of the problem space. Furthermore, the rate of gradient descent applied to each layer is tuned via self-adaptive mutation, thereby reducing the parameter optimisation task.

As someone with expertise in both real estate and data science, I've always been fascinated by Zillow's Zestimate. In the spirit of competition, I've developed Jim's estimate or Jestimate! The following interactive map contains 2018 home sales in San Francisco by neighborhood. Click on the neighborhood and then click on a home in the data table to see the Jestimate results versus the actual sales price. Zestimate uses a proprietary machine learning formula to estimate the current market value of a home.

I've been idly wondering lately about the problem of identifying high value samples to obtain for improving models, which seems to get at something similar under uncertainty. It isn't necessarily going to be economical to do an exhaustive sampling of whatever you're interested in, but collecting a few strategic datapoints could be relatively affordable and help a lot with inference. I also was wondering if some kind of hypothesis falsification module could be stapled onto gradient descent algorithms somehow. In terms of simulated annealing, because that's mentally easier for me, the idea would be that we want the temperature of nonlocal jumps to be hotter when the machine is making failed guesses, and we want it to be cooler when the gradient is behaving like the falsification module expects. The motivation for this is just that for inference, a lot of the time it is easier to learn things if you go out of your way to test your assumptions. Just having those assumptions be consistent with your observations is only a weak test of their value.

In a previous article, we've shown that a covariance matrix plot can be used for feature selection and dimensionality reduction: Feature Selection and Dimensionality Reduction Using Covariance Matrix Plot. We, therefore, were able to reduce the dimension of our feature space from 6 to 4. Now suppose we want to build a model on the new feature space for predicting the crew variable. Looking at the covariance matrix plot between features, we see that there is a strong correlation between the features (predictor variables), see the image above. In this article, we shall use a technique called Principal Component Analysis (PCA) to transform our features into space where the features are independent or uncorrelated. We shall then train our model on the PCA space.