This paper studies the prediction of chord progressions for jazz music by relying on machine learning models. The motivation of our study comes from the recent success of neural networks for performing automatic music composition. Although high accuracies are obtained in single-step prediction scenarios, most models fail to generate accurate multi-step chord predictions. In this paper, we postulate that this comes from the multi-scale structure of musical information and propose new architectures based on an iterative temporal aggregation of input labels. Specifically, the input and ground truth labels are merged into increasingly large temporal bags, on which we train a family of encoder-decoder networks for each temporal scale. In a second step, we use these pre-trained encoder bottleneck features at each scale in order to train a final encoder-decoder network. Furthermore, we rely on different reductions of the initial chord alphabet into three adapted chord alphabets. We perform evaluations against several state-of-the-art models and show that our multi-scale architecture outperforms existing methods in terms of accuracy and perplexity, while requiring relatively few parameters. We analyze musical properties of the results, showing the influence of downbeat position within the analysis window on accuracy, and evaluate errors using a musically-informed distance metric.

A BSTRACT In anomaly detection (AD), one seeks to identify whether a test sample is abnormal, given a data set of normal samples. A recent and promising approach to AD relies on deep generative models, such as variational autoencoders (V AEs), for unsupervised learning of the normal data distribution. In semi-supervised AD (SSAD), the data also includes a small sample of labeled anomalies. In this work, we propose two variational methods for training V AEs for SSAD. The intuitive idea in both methods is to train the encoder to'separate' between latent vectors for normal and outlier data. We show that this idea can be derived from principled probabilistic formulations of the problem, and propose simple and effective algorithms. Our methods can be applied to various data types, as we demonstrate on SSAD datasets ranging from natural images to astronomy and medicine, and can be combined with any V AE model architecture. When comparing to state-of-the-art SSAD methods that are not specific to particular data types, we obtain marked improvement in outlier detection. In its common formulation, training data is provided only for normal samples, while at test time, anomalous samples need to be detected. In the probabilistic AD approach, a model of the normal data distribution is learned, and the likelihood of a test sample under this model is thresholded for classification as normal or not. Recently, deep generative models such as variational autoencoders (V AEs, Kingma & Welling 2013) and generative adversarial networks (Goodfellow et al., 2014) have shown promise for learning data distributions in AD (An & Cho, 2015; Suh et al., 2016; Schlegl et al., 2017; Wang et al., 2017). Here, we consider the setting of semi-supervised AD (SSAD), where in addition to the normal samples, a small sample of labeled anomalies is provided (G ornitz et al., 2013). Most importantly, this set is too small to represent the range of possible anomalies, making classification methods (either supervised or semi-supervised) unsuitable. Instead, most approaches are based on'fixing' an unsupervised AD method to correctly classify the labeled anomalies, while still maintaining AD capabilities for unseen outliers (e.g., G ornitz et al., 2013; Mu noz-Mar ı et al., 2010; Ruff et al., 2019).

Automatic modulation classification (AMC) has been studied for more than a quarter of a century; however, it has been difficult to design a classifier that operates successfully under changing multipath fading conditions and other impairments. Recently, deep learning (DL)-based methods are adopted by AMC systems and major improvements are reported. In this paper, a novel convolutional neural network (CNN) classifier model is proposed to classify modulation classes in terms of their families, i.e., types. The proposed classifier is robust against realistic wireless channel impairments and in relation to that when the data sets that are utilized for testing and evaluating the proposed methods are considered, it is seen that RadioML2016.10a is the main dataset utilized for testing and evaluation of the proposed methods. However, the channel effects incorporated in this dataset and some others may lack the appropriate modeling of the real-world conditions since it only considers two distributions for channel models for a single tap configuration. Therefore, in this paper, a more comprehensive dataset, named as HisarMod2019.1, is also introduced, considering real-life applicability. HisarMod2019.1 includes 26 modulation classes passing through the channels with 5 different fading types and several numbers of taps for classification. It is shown that the proposed model performs better than the existing models in terms of both accuracy and training time under more realistic conditions. Even more, surpassed their performance when the RadioML2016.10a dataset is utilized.

Convolutional neural networks (CNNs) are deep learning fra meworks which are well-known for their notable performance in classification tasks. Hence, many skeleton-based action recognition and segmentation (SBAR S) algorithms benefit from them in their designs. However, a shortcoming of such applications is the general lack of spatial relationships between the input features in such data types. Besides, nonuniform temporal scalings is a common i ssue in skeleton-based data streams which leads to having different input siz es even within one specific action category. In this work, we propose a novel dee p-aligned convolu-tional neural network (DACNN) to tackle the above challenge s for the particular problem of SBARS. Our network is designed by introducing a ne w type of filters in the context of CNNs which are trained based on their alignm ents to the local subsequences in the inputs. These filters result in efficient predictions as well as learning interpretable patterns in the data. W e empiricall y evaluate our framework on real-world benchmarks showing that the proposed DACNN al gorithm obtains a competitive performance compared to the state-of-the-ar t while benefiting from a less complicated yet more interpretable model.

Abstract--Operating deep neural networks (DNNs) on devices with limited resources requires the reduction of their memo ry as well as computational footprint. Popular reduction method s are network quantization or pruning, which either reduce the wo rd length of the network parameters or remove weights from the network if they are not needed. In this article we discuss a ge neral framework for network reduction which we call Look-Up T able Quantization (LUT -Q). For each layer, we learn a value dictionary and an assignment matrix to represent the network weights. W e propose a special solver which combines gradient descent an d a one-step k-means update to learn both the value dictionari es and assignment matrices iteratively. This method is very fle xible: by constraining the value dictionary, many different reduc tion problems such as nonuniform network quantization, traini ng of multiplierless networks, network pruning or simultaneo us quantization and pruning can be implemented without changi ng the solver . This flexibility of the LUT -Q method allows us to use the same method to train networks for different hardware capabilities. Deep neural networks (DNN)s are currently used in many machine learning and signal processing applications with g reat success as their performance often beats the previous state - of-the-art approaches by a large margin, e.g., see [2] for an overview of deep learning. DNN approaches have become standard practice in computer vision, automatic speech rec og-nition and partially in natural language processing. They a re also extensively investigated to support other domains lik e medicine, robotics and finance forecasting. Recently, there has been a lot of interest in the research community in reducing the memory/computational footprint of neural networks.

Due to the increasing integration of solar power into the electrical grid, forecasting short-term solar irradiance has become key for many applications, e.g. In this context, as solar generators are geographically dispersed and ground measurements are not always easy to obtain, it is very important to have general models that can predict solar irradiance without the need of local data. In this paper, a model that can perform short-term forecasting of solar irradiance in any general location without the need of ground measurements is proposed. To do so, the model considers satellite-based measurements and weather-based forecasts, and employs a deep neural network structure that is able to generalize across locations; particularly, the network is trained only using a small subset of sites where ground data is available, and the model is able to generalize to a much larger number of locations where ground data does not exist. As a case study, 25 locations in The Netherlands are considered and the proposed model is compared against four local models that are individually trained for each location using ground measurements. Despite the general nature of the model, it is shown show that the proposed model is equal or better than the local models: when comparing the average performance across all the locations and prediction horizons, the proposed model obtains a 31.31% Introduction With the increasing integration of renewable sources into the electrical grid, accurate forecasting of renewable source generation has become one of the most important challenges across several applications. Among them, balancing the electrical grid via activation of reserves is arguably one of the most critical ones to ensure a stable system. In particular, due to their intermittent and unpredictable nature, the more renewables are integrated, the more complex the grid management becomes [1, 2]. This is the postprint of the article: Short-term forecasting of solar irradiance without local telemetry: a generalized model using satellite data, Solar Energy 173 (2018), 566-577 . Corresponding author Email address: j.lagogarcia@tudelft.nl (Jesus Lago) In particular, in addition to activation of reserves to manage the grid stability, short-term forecasts of solar irradiance are paramount for operational planning, switching sources, programming backup, short-term power trading, peak load matching, scheduling of power systems, congestion management, and cost reduction [2-4]. Solar irradiance forecasting The forecasting of solar irradiance can be typically divided between methods for global horizontal irradiance (GHI) and methods for direct normal irradiance (DNI) [5], with the latter being a component of the GHI (together with the diffuse solar irradiance). As in this work GHI is forecasted, [5] should be used for a complete review on methods for DNI.

Electrocardiogram signals are omnipresent in medicine. A vital aspect in the analysis of this data is the identification and classification of heart beat types which is often done through automated algorithms. Advancements in neural networks and deep learning have led to a high classification accuracy. However, the final adoption of these models into clinical practice is limited due to the black-box nature of the methods. In this work, we explore the use of variational auto-encoders based on linear dense networks to learn human interpretable beat embeddings in time-series data. We demonstrate that using this method, an interpretable and explainable ECG beat space can be generated, set up by characteristic base beats.

To accelerate the existing Broad Learning System (BLS) for new added nodes in [7], we extend the inverse Cholesky factorization in [10] to deduce an efficient inverse Cholesky factorization for a Hermitian matrix partitioned into 2 * 2 blocks, which is utilized to develop the proposed BLS algorithm 1. The proposed BLS algorithm 1 compute the ridge solution (i.e, the output weights) from the inverse Cholesky factor of the Hermitian matrix in the ridge inverse, and update the inverse Cholesky factor efficiently. From the proposed BLS algorithm 1, we deduce the proposed ridge inverse, which can be obtained from the generalized inverse in [7] by just change one matrix in the equation to compute the newly added sub-matrix. We also modify the proposed algorithm 1 into the proposed algorithm 2, which is equivalent to the existing BLS algorithm [7] in terms of numerical computations. The proposed algorithms 1 and 2 can reduce the computational complexity, since usually the Hermitian matrix in the ridge inverse is smaller than the ridge inverse. With respect to the existing BLS algorithm, the proposed algorithms 1 and 2 usually require about 13 and 2 3 of complexities, respectively, while in numerical experiments they achieve the speedups (in each additional training time) of 2.40 - 2.91 and 1.36 - 1.60, respectively. Numerical experiments also show that the proposed algorithm 1 and the standard ridge solution always bear the same testing accuracy, and usually so do the proposed algorithm 2 and the existing BLS algorithm. The existing BLS assumes the ridge parameter lamda->0, since it is based on the generalized inverse with the ridge regression approximation. When the assumption of lamda-> 0 is not satisfied, the standard ridge solution obviously achieves a better testing accuracy than the existing BLS algorithm in numerical experiments.

The inverse-free extreme learning machine (ELM) algorithm proposed in [4] was based on an inverse-free algorithm to compute the regularized pseudo-inverse, which was deduced from an inverse-free recursive algorithm to update the inverse of a Hermitian matrix. Before that recursive algorithm was applied in [4], its improved version had been utilized in previous literatures [9], [10]. Accordingly from the improved recursive algorithm [9], [10], we deduce a more efficient inverse-free algorithm to update the regularized pseudo-inverse, from which we develop the proposed inverse-free ELM algorithm 1. Moreover, the proposed ELM algorithm 2 further reduces the computational complexity, which computes the output weights directly from the updated inverse, and avoids computing the regularized pseudoinverse. Lastly, instead of updating the inverse, the proposed ELM algorithm 3 updates the LDLT factor of the inverse by the inverse LDLT factorization [11], to avoid numerical instabilities after a very large number of iterations [12]. With respect to the existing ELM algorithm, the proposed ELM algorithms 1, 2 and 3 are expected to require only (8+3)/M , (8+1)/M and (8+1)/M of complexities, respectively, where M is the output node number. In the numerical experiments, the standard ELM, the existing inverse-free ELM algorithm and the proposed ELM algorithms 1, 2 and 3 achieve the same performance in regression and classification, while all the 3 proposed algorithms significantly accelerate the existing inverse-free ELM algorithm

In this paper, we propose a new wrapper approach for semi-supervised feature selection. A common strategy in semi-supervised learning is to augment the training set by pseudo-labeled unlabeled examples. However, the pseudo-labeling procedure is prone to error and has a high risk of disrupting the learning algorithm with additional noisy labeled training data. To overcome this, we propose to model explicitly the mislabeling error during the learning phase with the overall aim of selecting the most relevant feature characteristics. We derive a $\mathcal{C}$-bound for Bayes classifiers trained over partially labeled training sets by taking into account the mislabeling errors. The risk bound is then considered as an objective function that is minimized over the space of possible feature subsets using a genetic algorithm. In order to produce both sparse and accurate solution, we propose a modification of a genetic algorithm with the crossover based on feature weights and recursive elimination of irrelevant features. Empirical results on different data sets show the effectiveness of our framework compared to several state-of-the-art semi-supervised feature selection approaches.