In the design of brain-computer interface systems, classification of Electroencephalogram (EEG) signals is the essential part and a challenging task. Recently, as the marginalized discrete wavelet transform (mDWT) representations can reveal features related to the transient nature of the EEG signals, the mDWT coefficients have been frequently used in EEG signal classification. In our previous work, we have proposed a super-Dirichlet distribution-based classifier, which utilized the nonnegative and sum-to-one properties of the mDWT coefficients. The proposed classifier performed better than the state-of-the-art support vector machine-based classifier. In this paper, we further study the neutrality of the mDWT coefficients. Assuming the mDWT vector coefficients to be a neutral vector, we transform them non-linearly into a set of independent scalar coefficients. Feature selection strategy is proposed on the transformed feature domain. Experimental results show that the feature selection strategy helps improving the classification accuracy.

When approaching any type of Machine Learning (ML) problem there are many different algorithms to choose from. In machine learning, there's something called the "No Free Lunch" theorem which basically states that no one ML algorithm is best for all problems. The performance of different ML algorithms strongly depends on the size and structure of your data. Thus, the correct choice of algorithm often remains unclear unless we test out our algorithms directly through plain old trial and error. But, there are some pros and cons to each ML algorithm that we can use as guidance.

Which machine learning algorithm should I use?

The purpose of this post is to unpack to the layman the basic concepts of applied machine learning and to document how data scientists or data analysts would generally answer a question or solve a problem with data and machine learning algorithms. Hopefully, by the end, you would have a more solid understanding of the steps your data scientist or business intelligence officers should be going through when attempting to apply the power of machine learning to data. Machine learning is a method of data analysis that automates analytical model building. The steps illustrated here are written as a'practical guide' of that method. It covers the broad strokes of the process one would go through when implementing any other similar machine learning algorithms or ideas.

This is the 3rd article of series "Coding Deep Learning for Beginners". Here, you will be able to find links to all articles, agenda, and general information about an estimated release date of next articles on the bottom of the 1st article. They are also available in my open source portfolio -- MyRoadToAI, along with some mini-projects, presentations, tutorials and links. You can also read the article on my personal website, hosted with Jekyll in order to improve readability (supporting code syntax highlighting, LaTeX equations and more. Some of you may wonder, why the article series about explaining and coding Neural Networks starts with basic Machine Learning algorithm such as Linear Regression.

Note: This article will likely be revised and expanded before being submitted for review and publication. At the moment it is missing critical sections, that will be added later. If we have suggestions for improvement, please send them to me directly. In this article I will discuss the well-known technique of feature hashing, but with the modification of performing the hashing step on the client-side before sending data to a server or daemon performing model training and prediction. By using this approach, we can ensure that the system performing the training cannot have any knowledge of the underlying data being received, since the learning takes place only using the hashed representation of the data.

Completing a data matrix X has become an ubiquitous problem in modern data science, with applications in recommender systems, computer vision, and networks inference, to name a few. One typical assumption is that X is low-rank. A more general model assumes that each column of X corresponds to one of several low-rank matrices. This paper generalizes these models to what we call mixture matrix completion (MMC): the case where each entry of X corresponds to one of several low-rank matrices. MMC is a more accurate model for recommender systems, and brings more flexibility to other completion and clustering problems. We make four fundamental contributions about this new model. First, we show that MMC is theoretically possible (well-posed). Second, we give its precise information-theoretic identifiability conditions. Third, we derive the sample complexity of MMC. Finally, we give a practical algorithm for MMC with performance comparable to the state-of-the-art for simpler related problems, both on synthetic and real data.

Statistical shape models enhance machine learning algorithms providing prior information about deformation. A Point Distribution Model (PDM) is a popular landmark-based statistical shape model for segmentation. It requires choosing a model order, which determines how much of the variation seen in the training data is accounted for by the PDM. A good choice of the model order depends on the number of training samples and the noise level in the training data set. Yet the most common approach for choosing the model order simply keeps a predetermined percentage of the total shape variation. In this paper, we present a technique for choosing the model order based on information-theoretic criteria, and we show empirical evidence that the model order chosen by this technique provides a good trade-off between over- and underfitting.

For the published version of the article, see: Heck, D. W., Moshagen, M., & Erdfelder, E. (2014). Correspondence concerning this article should be addressed to Daniel W. Heck, Department of Psychology, School of Social Sciences, University of Mannheim, Schloss EO 254, D-68131 Mannheim, Germany. FISHER INFORMATION APPROXIMATION 2 Abstract The Fisher information approximation (FIA) is an implementation of the minimum description length principle for model selection. Unlike information criteria such as AIC or BIC, it has the advantage of taking the functional form of a model into account. Unfortunately, FIA can be misleading in finite samples, resulting in an inversion of the correct rank order of complexity terms for competing models in the worst case.

In the absence of explicit regularization, Kernel "Ridgeless" Regression with nonlinear kernels has the potential to fit the training data perfectly. It has been observed empirically, however, that such interpolated solutions can still generalize well on test data. We isolate a phenomenon of implicit regularization for minimum-norm interpolated solutions which is due to a combination of high dimensionality of the input data, curvature of the kernel function, and favorable geometric properties of the data such as an eigenvalue decay of the empirical covariance and kernel matrices. In addition to deriving a data-dependent upper bound on the out-of-sample error, we present experimental evidence suggesting that the phenomenon occurs in the MNIST dataset.