In mid-September the European Union threatened to fine the Big Tech companies if they did not remove terrorist content within one hour of appearing online. The change came because rising tensions are now developing and being played out on social media platforms. Social conflicts that once built up in backroom meetings and came to a head on city streets, are now building momentum on social media platforms before spilling over into real life. In the past, governments tended to control traditional media, with little to no possibility for individuals to broadcast hate. The digital revolution has altered everything.

Considering music as a sequence of events with multiple complex dependencies, the Long Short-Term Memory (LSTM) architecture has proven very efficient in learning and reproducing musical styles. However, the generation of rhythms requires additional information regarding musical structure and accompanying instruments. In this paper we present DeepDrum, an adaptive Neural Network capable of generating drum rhythms under constraints imposed by Feed-Forward (Conditional) Layers which contain musical parameters along with given instrumentation information (e.g. bass and guitar notes). Results on generated drum sequences are presented indicating that DeepDrum is effective in producing rhythms that resemble the learned style, while at the same time conforming to given constraints that were unknown during the training process.

I implemented the Neural Processes paper in Pytorch, and found a few interesting things not discussed elsewhere as far as I can see. For example, the posterior is liable to collapse to a near point estimate, and this appears to be somewhat related to the formulation they use. I wrote this up in an informal blog post, thoughts/comments welcome!

He said the main part of its RL policy is implemented by a 4,000 dimensional LSTM network, which has roughly 100 million (10 9) parameters .He also said that in terms of Flops, it is like a honeybee brain. This means the OpenAI Dota2 bot has roughly the same number of parameters as honeybee brain (Assumption: real synapse weight can be represented by 1 Byte of memory, which is often the case in artificial neural network systems). If the former is more intelligent, it suggests that modern machine learning algorithm is more capable than natural selection at turning a certain size of parameters into intelligence. It is probable that the same algorithm can turn them into super-mice or super-cat intelligence.

I think that eager sucks. It does not fit my mental model. I am just worried that TF wants to attract PyTorch users, but a lot of the TF users actually prefer the current state.

This means that certain aspects of PyTorch are hidden for convenience. This makes certain routines easier and adds additional functionality but introduces an additional layer of abstraction. This series starts with PyTorch at the bottom and moves upward (bottom up approach), so it's really a matter of preference for both the course and the library. The general suggestion is to use both courses as learning resources, and to learn pure PyTorch as well as the fast.ai

If you know any particularly good (or bad!) visualizations, I encourage you to share it here! Also, if you have a clear vision for an automatic tool for creating publication-ready diagrams, I am share it with us (can be "TensorBoard ..."). That is - what is important yo you that is missing in the existing tools?

TL,DR: We're not sure yet what this means for a neural net user. But, there is a connection to RMSProp and Adam, which we explore in the preprint (see the relativistic kinetic energy subsection). Given that the kinetic energy is a design choice, there's lots to explore. Keep in mind that these methods are generalizations of the momentum method, which is one of the more popular neural network optimizers. In the preprint we very briefly consider conditions that guarantee convergence to stationary points for non-convex (i.e., neural nets) functions.

I'm currently writing a series of blog posts aimed at demystifying the maths behind deep learning. I've gone through the intuition as well as step-by-step through the maths, and I've also got code samples for you to implement the neural networks from scratch. Hope you find the posts useful, please comment if there are parts that could be made clearer. I'll be posting the RNN/LSTM backprop posts in a few days, in the meantime I have already posted the CNN and neural network backprop posts if you want to check that out.

If you aim to become deep learning researcher then you should lean things more deeply then just their implementation part. I would advice to start working on some project along with the reading stuff. A good knowledge of these concepts is required for reading research paper. Then learn basic ML stuff and deep learning concept. Doing this course will give you sufficient knowledge about the basic architectures of deep learning.