It has been a long-standing problem to efficiently learn a halfspace using as few labels as possible in the presence of noise. In this work, we propose an efficient Perceptron-based algorithm for actively learning homogeneous halfspaces under the uniform distribution over the unit sphere. Under the adversarial noise condition \cite{ABL14, KLS09, KKMS08}, where at most a $\tilde \Omega(\epsilon)$ fraction of labels can be flipped, our algorithm achieves a near-optimal label complexity of $\tilde{O}\left(d\ln\frac{1}{\epsilon}\right)$ in time $\tilde{O}\left(\frac{d 2}{\epsilon}\right)$. Furthermore, we show that our active learning algorithm can be converted to an efficient passive learning algorithm that has near-optimal sample complexities with respect to $\epsilon$ and $d$. Papers published at the Neural Information Processing Systems Conference.

Various factors, such as identities, views (poses), and illuminations, are coupled in face images. Disentangling the identity and view representations is a major challenge in face recognition. Existing face recognition systems either use handcrafted features or learn features discriminatively to improve recognition accuracy. This is different from the behavior of human brain. Intriguingly, even without accessing 3D data, human not only can recognize face identity, but can also imagine face images of a person under different viewpoints given a single 2D image, making face perception in the brain robust to view changes.

Wind power as a renewable source of energy, has numerous economic, environmental and social benefits. In order to enhance and control renewable wind power, it is vital to utilize models that predict wind speed with high accuracy. Due to neglecting of requirement and significance of data preprocessing and disregarding the inadequacy of using a single predicting model, many traditional models have poor performance in wind speed prediction. In the current study, for predicting wind speed at target stations in the north of Iran, the combination of a multi-layer perceptron model (MLP) with the Whale Optimization Algorithm (WOA) used to build new method (MLP-WOA) with a limited set of data (2004-2014). Then, the MLP-WOA model was utilized at each of the ten target stations, with the nine stations for training and tenth station for testing (namely: Astara, Bandar-E-Anzali, Rasht, Manjil, Jirandeh, Talesh, Kiyashahr, Lahijan, Masuleh, and Deylaman) to increase the accuracy of the subsequent hybrid model. The capability of the hybrid model in wind speed forecasting at each target station was compared with the MLP model without the WOA optimizer. To determine definite results, numerous statistical performances were utilized. For all ten target stations, the MLP-WOA model had precise outcomes than the standalone MLP model. The hybrid model had acceptable performances with lower amounts of the RMSE, SI and RE parameters and higher values of NSE, WI, and KGE parameters. It was concluded that the WOA optimization algorithm can improve the prediction accuracy of MLP model and may be recommended for accurate wind speed prediction.

Graph generation is an extremely important task, as graphs are found throughout different areas of science and engineering. In this work, we focus on the modern equivalent of the Erdos-Renyi random graph model: the graph variational autoencoder (GVAE). This model assumes edges and nodes are independent in order to generate entire graphs at a time using a multi-layer perceptron decoder. As a result of these assumptions, GVAE has difficulty matching the training distribution and relies on an expensive graph matching procedure. We improve this class of models by building a message passing neural network into GVAE's encoder and decoder. We demonstrate our model on the specific task of generating small organic molecules

How far apart are two neural networks? This is a foundational question in their theory. We derive a simple and tractable bound that relates distance in function space to distance in parameter space for a broad class of nonlinear compositional functions. The bound distills a clear dependence on depth of the composition. The theory is of practical relevance since it establishes a trust region for first-order optimisation. In turn, this suggests an optimiser that we call Frobenius matched gradient descent---or Fromage. Fromage involves a principled form of gradient rescaling and enjoys guarantees on stability of both the spectra and Frobenius norms of the weights. We find that the new algorithm increases the depth at which a multilayer perceptron may be trained as compared to Adam and SGD and is competitive with Adam for training generative adversarial networks. We further verify that Fromage scales up to a language transformer with over $10^8$ parameters. Please find code & reproducibility instructions at: https://github.com/jxbz/fromage.

Group invariant and equivariant Multilayer Perceptrons (MLP), also known as Equivariant Networks, have achieved remarkable success in learning on a variety of data structures, such as sequences, images, sets, and graphs. Using tools from group theory, this paper proves the universality of a broad class of equivariant MLPs with a single hidden layer. In particular, it is shown that having a hidden layer on which the group acts regularly is sufficient for universal equivariance. Next, Burnside's table of marks is used to decompose product spaces. It is shown that the product of two G-sets always contains an orbit larger than the input orbits. Therefore high order hidden layers inevitably contain a regular orbit, leading to the universality of the corresponding MLP. It is shown that with an order larger than the logarithm of the size of the stabilizer group, a high-order equivariant MLP is equivariant universal.

Deep learning and neural networks are very interesting subjects and Go language supports this technology using the framework Gorgonia. Gorgonia works like similar frameworks, for example, Tensorflow, allowing the creation of graphs of operations and tensors. There isn't something like Keras for Gorgonia, although the project Golgi is promising. The programming is a little low level, because we have to create the graphs containing the operations and the tensors, so there is no concept of neuron or layers, which exists in other frameworks. In this post, I will show a very basic example of MLP - Multilayer Perceptron trying to find the weights for an approximation of the exclusive disjunction function or XOR.

Optical Wireless Communication (OWC) propagation channel characterization plays a key role on the design and performance analysis of Vehicular Visible Light Communication (VVLC) systems. Current OWC channel models based on deterministic and stochastic methods, fail to address mobility induced ambient light, optical turbulence and road reflection effects on channel characterization. Therefore, alternative machine learning (ML) based schemes, considering ambient light, optical turbulence, road reflection effects in addition to intervehicular distance and geometry, are proposed to obtain accurate VVLC channel loss and channel frequency response (CFR). This work demonstrates synthesis of ML based VVLC channel model frameworks through multi layer perceptron feed-forward neural network (MLP), radial basis function neural network (RBF-NN) and Random Forest ensemble learning algorithms. Predictor and response variables, collected through practical road measurements, are employed to train and validate proposed models for various conditions. Additionally, the importance of different predictor variables on channel loss and CFR is assessed, normalized importance of features for measured VVLC channel is introduced. We show that RBF-NN, Random Forest and MLP based models yield more accurate channel loss estimations with 3.53 dB, 3.81 dB, 3.95 dB root mean square error (RMSE), respectively, when compared to fitting curve based VVLC channel model with 7 dB RMSE. Moreover, RBF-NN and MLP models are demonstrated to predict VVLC CFR with respect to distance, ambient light and receiver inclination angle predictor variables with 3.78 dB and 3.60 dB RMSE respectively.

We consider building predictors when the data have missing values. We study the seemingly-simple case where the target to predict is a linear function of the fully-observed data and we show that, in the presence of missing values, the optimal predictor may not be linear. In the particular Gaussian case, it can be written as a linear function of multiway interactions between the observed data and the various missing-value indicators. Due to its intrinsic complexity, we study a simple approximation and prove generalization bounds with finite samples, highlighting regimes for which each method performs best. We then show that multilayer perceptrons with ReLU activation functions can be consistent, and can explore good trade-offs between the true model and approximations. Our study highlights the interesting family of models that are beneficial to fit with missing values depending on the amount of data available.

Syntactic parsing using dependency structures has become a standard technique in natural language processing with many different parsing models, in particular data-driven models that can be trained on syntactically annotated corpora. In this paper, we tackle transition-based dependency parsing using a Perceptron Learner. Our proposed model, which adds more relevant features to the Perceptron Learner, outperforms a baseline arc-standard parser. We beat the UAS of the MALT and LSTM parsers. We also give possible ways to address parsing of non-projective trees.