Goto

Collaborating Authors

 Deep Learning


Music Style Classification with Compared Methods in XGB and BPNN

arXiv.org Machine Learning

--Scientists have used many different classification methods to solve the problem of music classification. But the efficiency of each classification is different. In this paper, we propose two compared methods on the task of music style classification. More specifically, feature extraction for representing timbral texture, rhythmic content and pitch content are proposed. Comparative evaluations on performances of two classifiers were conducted for music classification with different composers' styles.


Towards Understanding the Spectral Bias of Deep Learning

arXiv.org Machine Learning

An intriguing phenomenon observed during training neural networks is the spectral bias, where neural networks are biased towards learning less complex functions. The priority of learning functions with low complexity might be at the core of explaining generalization ability of neural network, and certain efforts have been made to provide theoretical explanation for spectral bias. However, there is still no satisfying theoretical result justifying the underlying mechanism of spectral bias. In this paper, we give a comprehensive and rigorous explanation for spectral bias and relate it with the neural tangent kernel function proposed in recent work. We prove that the training process of neural networks can be decomposed along different directions defined by the eigenfunctions of the neural tangent kernel, where each direction has its own convergence rate and the rate is determined by the corresponding eigenvalue. We then provide a case study when the input data is uniformly distributed over the unit sphere, and show that lower degree spherical harmonics are easier to be learned by over-parameterized neural networks.


A binary-activation, multi-level weight RNN and training algorithm for processing-in-memory inference with eNVM

arXiv.org Machine Learning

We present a new algorithm for training neural networks with binary activations and multi-level weights, which enables efficient processing-in-memory circuits with eNVM. Binary activations obviate costly DACs and ADCs. Multi-level weights leverage multi-level eNVM cells. Compared with previous quantization algorithms, our method not only works for feed-forward networks including fully-connected and convolutional, but also achieves higher accuracy and noise resilience for recurrent networks. In particular, we present a RNN trigger-word detection PIM accelerator, whose modeling results demonstrate high performance using our new training algorithm.


ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks

arXiv.org Artificial Intelligence

We present ALFRED (Action Learning From Realistic Environments and Directives), a benchmark for learning a mapping from natural language instructions and egocentric vision to sequences of actions for household tasks. Long composition rollouts with non-reversible state changes are among the phenomena we include to shrink the gap between research benchmarks and real-world applications. ALFRED consists of expert demonstrations in interactive visual environments for 25k natural language directives. These directives contain both high-level goals like "Rinse off a mug and place it in the coffee maker." and low-level language instructions like "Walk to the coffee maker on the right." ALFRED tasks are more complex in terms of sequence length, action space, and language than existing vision-and-language task datasets. We show that a baseline model designed for recent embodied vision-and-language tasks performs poorly on ALFRED, suggesting that there is significant room for developing innovative grounded visual language understanding models with this benchmark.


Dream to Control: Learning Behaviors by Latent Imagination

arXiv.org Artificial Intelligence

Learned world models summarize an agent's experience to facilitate learning complex behaviors. While learning world models from high-dimensional sensory inputs is becoming feasible through deep learning, there are many potential ways for deriving behaviors from them. We present Dreamer, a reinforcement learning agent that solves long-horizon tasks from images purely by latent imagination. We efficiently learn behaviors by propagating analytic gradients of learned state values back through trajectories imagined in the compact state space of a learned world model. On 20 challenging visual control tasks, Dreamer exceeds existing approaches in data-efficiency, computation time, and final performance.


BADGER: Learning to (Learn [Learning Algorithms] through Multi-Agent Communication)

arXiv.org Artificial Intelligence

An architecture and a learning procedure where: An agent is made up of many experts All experts share the same communication policy (expert policy), but have different internal memory states There are two levels of learning, an inner loop (with a communication stage) and an outer lo op In ner loop - Agent's behavior and adaptation should emerge as a result of e xperts communicating between each other. Expert s send messag es (of any complexity) to each other and update their internal states based on observations/messages and their internal state fr om the previous time-step. Expert policy is fixed and does not c hange during the inner loop Inner loop loss need not even be a proper loss function. It can be any kind of structured feedback guiding the adaptation during th e age nt's lifetime Outer loop - An expert policy is discovered over generations of agents, ensuring that strategies that find solutions to prob lems in divers e environments can quickly emerge in the inner loop Agent's objective is to adapt fast to novel tasks Exhibiting the following novel properties: Roles of experts and connectivity among them assigned dynamically at inference time Learned communication protocol with context dependent messages of varied complexity Generalizes to different numbers and types of inputs/ou tputs Ca n be trained to handle variations in architecture during bot h training and testing Initial empirical results show generalization and scalability along the spectrum of learning types.


Integrating Graph Contextualized Knowledge into Pre-trained Language Models

arXiv.org Artificial Intelligence

Complex node interactions are common in knowledge graphs, and these interactions also contain rich knowledge information. However, traditional methods usually treat a triple as a training unit during the knowledge representation learning (KRL) procedure, neglecting contextualized information of the nodes in knowledge graphs (KGs). We generalize the modeling object to a very general form, which theoretically supports any subgraph extracted from the knowledge graph, and these subgraphs are fed into a novel transformer-based model to learn the knowledge embeddings. To broaden usage scenarios of knowledge, pre-trained language models are utilized to build a model that incorporates the learned knowledge representations. Experimental results demonstrate that our model achieves the state-of-the-art performance on several medical NLP tasks, and improvement above TransE indicates that our KRL method captures the graph contextualized information effectively.


AWS announces DeepComposer, a machine-learning keyboard for developers – TechCrunch

#artificialintelligence

Today, as AWS re:Invent begins, Amazon announced DeepComposer, a machine learning-driven keyboard aimed at developers. "AWS DeepComposer is a 32-key, 2-octave keyboard designed for developers to get hands on with Generative AI, with either pretrained models or your own," AWS' Julien Simon wrote in a blog post introducing the company's latest machine learning hardware. The keyboard is supposed to help developers learn about machine learning in a fun way, and maybe create some music along the way. The area involved in generating creative works in artificial intelligence is called "generative AI." In other words, it helps you teach machines to generate something creative using "generative adversarial networks."


How might an AI explain itself?

#artificialintelligence

In his blog post on artificial intelligence (AI), GovTech Graduate Jonathan Manning draws on the New Zealand Law Foundation: Government use of artificial intelligence in New Zealand (the NZFL report) to discuss the role and effectiveness of explanation tools. As algorithms and AI become ubiquitous we all become'data subjects' to organisations such as governments and businesses. In response, regulations such as the EU's General Data Protection Regulation are beginning to emerge. The New Zealand government is currently exploring how governments, business and society can work together to meet the challenge of regulating AI. A part of this challenge is ensuring when things like algorithmic harm arise, we can explain what happened and why so that mistakes can be fixed and not repeated or obscured.


Deep Learning: Past, Present, and Future…

#artificialintelligence

Deep Learning is a subset of Artificial Intelligence that falls under the umbrella of machine learning. It is inspired by the structure and function of the human brain. Deep Learning is a method of enabling computers to carry out specific tasks that require human intelligence without any human intervention. The human brain consists of a complex web of neurons connected together, this network of neurons (also called neural network) is responsible for complex functionalities of the human brain. Deep Learning was built on the core idea of mimicking the complex process of human brain to enable machines to think on their own i.e. machines with artificial intelligence. Artificial neural networks were at the center of the deep learning revolution, these algorithms are loosely based on the structure of biological brains, which consist of networks of neurons interconnected by signal-carrying synapses.