Collaborating Authors

Sequence Transduction with Recurrent Neural Networks Machine Learning

Many machine learning tasks can be expressed as the transformation---or \emph{transduction}---of input sequences into output sequences: speech recognition, machine translation, protein secondary structure prediction and text-to-speech to name but a few. One of the key challenges in sequence transduction is learning to represent both the input and output sequences in a way that is invariant to sequential distortions such as shrinking, stretching and translating. Recurrent neural networks (RNNs) are a powerful sequence learning architecture that has proven capable of learning such representations. However RNNs traditionally require a pre-defined alignment between the input and output sequences to perform transduction. This is a severe limitation since \emph{finding} the alignment is the most difficult aspect of many sequence transduction problems. Indeed, even determining the length of the output sequence is often challenging. This paper introduces an end-to-end, probabilistic sequence transduction system, based entirely on RNNs, that is in principle able to transform any input sequence into any finite, discrete output sequence. Experimental results for phoneme recognition are provided on the TIMIT speech corpus.

Converting DNA Sequence to Protein Sequence using Deep Neural Network with Interactive Code [Manual…


So today, I will continue my journey to Bio-informatics with Machine Learning. And I will try to perform the most basic task in Bio-informatics, which is converting DNA sequence to Protein. Also, this is over complicating the task, we can just build a dictionary to map the values, as done by Vijini Mallawaarachchi in this post. Also, please take note that we are going to preprocess the DNA / Protein sequence to vectors, if you are not aware of how to do that, please see this post. Finally, I am going to perform Dilated Back Propagation to train our network.

r/MachineLearning - [Discussion] Google Patents "Generating output sequences from input sequences using neural networks"


Between this, the GAN evaluation paper which happened to be really similar to a previously published paper by other authors, and DeepMind's PR machine while lacking in exhibiting the crucial details which make their Go models so good, I am definitely more and more disappointed in DeepMind ...

Meta Learning with Relational Information for Short Sequences

Neural Information Processing Systems

This paper proposes a new meta-learning method -- named HARMLESS (HAwkes Relational Meta Learning method for Short Sequences) for learning heterogeneous point process models from a collection of short event sequence data along with a relational network. Specifically, we propose a hierarchical Bayesian mixture Hawkes process model, which naturally incorporates the relational information among sequences into point process modeling. Compared with existing methods, our model can capture the underlying mixed-community patterns of the relational network, which simultaneously encourages knowledge sharing among sequences and facilitates adaptively learning for each individual sequence. We further propose an efficient stochastic variational meta-EM algorithm, which can scale to large problems. Numerical experiments on both synthetic and real data show that HARMLESS outperforms existing methods in terms of predicting the future events.

Sequence prediction using recurrent neural networks(LSTM) with TensorFlow


This post tries to demonstrates how to approximate a sequence of vectors using a recurrent neural networks, in particular I will be using the LSTM architecture, The complete code used for this post could be found here. Most of the examples I found in the internet apply the LSTM architecture to natural language processing problems, and I couldn't find an example where this architecture could be used to predict continuous values. So the task here is to predict a sequence of real numbers based on previous observations. The traditional neural networks architectures can't do this, this is why recurrent neural networks were made to address this issue, as they allow to store previous information to predict future event. Next we need to prepare the data in a way that could be accepted by our model.