Inattentional blindness is the psychological phenomenon that causes one to miss things in plain sight. It is a consequence of the selective attention in perception that lets us remain focused on important parts of our world without distraction from irrelevant details. Motivated by selective attention, we study the properties of artificial agents that perceive the world through the lens of a self-attention bottleneck. By constraining access to only a small fraction of the visual input, we show that their policies are directly interpretable in pixel space. We find neuroevolution ideal for training self-attention architectures for vision-based reinforcement learning (RL) tasks, allowing us to incorporate modules that can include discrete, non-differentiable operations which are useful for our agent.

I'm going to do a comparison on recent (or at least lesser-known) gradient optimization methods. The ones I've encountered up to now are the following: I would be particularly interested in approaches not using any hyperparameters at all (such as number 3 - COCOB), however I will consider all of the interesting and promising methods. Are you aware of some novelties or lesser-known approaches? Previously I've posted the question in r/MLQuestions but haven't received any feedback so I'm posting it here, hope it's not violating any rules.

Although the existing guide is very good, it lacks a good explanation of the Decoder side. It simply says that it is very similar to the Encoder without really provide much detail. In my eyes, the whole "magic" of the transformer is the ability to have different input and output lengths! That part is poorly explained, simply saying that the Decoder is using the Key and Value of the Encoder is not enough because it doesn't explain how the different dimensions are mitigated.

Abstract: Many existing conditional Generative Adversarial Networks (cGANs) are limited to conditioning on pre-defined and fixed class-level semantic labels or attributes. We propose an open set GAN architecture (OpenGAN) that is conditioned per-input sample with a feature embedding drawn from a metric space. Using a state-of-the-art metric learning model that encodes both class- level and fine-grained semantic information, we are able to generate samples that are semantically similar to a given source image. The semantic information extracted by the metric learning model transfers to out-of- distribution novel classes, allowing the generative model to produce samples that are outside of the training distribution. We show that our proposed method is able to generate 256$\times$256 resolution images from novel classes that are of similar visual quality to those from the training classes.

I have to classify a point(0 or 1) by its position in image and a pre calculated score. I was thinking to a discriminative classifier. Do you have any idea of model that I could try, maybe a neural network?

At each time step, I need to give a score to a varying number of items n. Depending on the score some different things will happen to the i best and j worst items. I need to optimize the neural network that does the scoring through time.

Nassim Nicholas Taleb, in his book "The Bed of Procrustes" noted that "people focus on role models; it is more effective to find anti-models -- people you don't want to resemble when you grow up." While Taleb's quote is commonly referenced across journalism and social media today, it isn't as applicable to real life as we might think -- at least not to real business life. Rather than seeking out contrarian voices, industry leaders tend to be overly influenced by mainstream thinking. Consider all the searches for those SEO-optimized webpages that perennially comprise page-one search results, reinforcing the same timeworn, made-for-PowerPoint sets of ideas over and over again. "Establish a data lake," they instruct business users. It is staggering to consider the number of companies fundamentally built on such "best practices", their leaders devoutly believing that in doing so they will roll out inevitably trend-setting products and features and win the battle for customers. Thinking in the fields of Data Sciences and Artificial Intelligence (AI) has been no different. In fact, "machine learning" has been the poster child of enterprise success stories for the last decade.

Oftentimes, we are surprised by the accuracy of recommendations on what to buy on Amazon, watch on Netflix, or listen on Spotify. We feel that somehow these companies know how our brain works and monetizing this magical guessing game. They have a deep foundation on behavioral sciences, and our job is to make all these concepts real in a way that is both easy to understand and covers the most important concepts. Remember: People are extremely predictable. Personality builds our conduct and our conduct determines our decisions.

Recent progress in natural language generation has raised dual-use concerns. While applications like summarization and translation are positive, the underlying technology also might enable adversaries to generate neural fake news: targeted propaganda that closely mimics the style of real news. Modern computer security relies on careful threat modeling: identifying potential threats and vulnerabilities from an adversary's point of view, and exploring potential mitigations to these threats. Likewise, developing robust defenses against neural fake news requires us first to carefully investigate and characterize the risks of these models. We thus present a model for controllable text generation called Grover.

Music source separation has been a popular topic in signal processing for decades, not only because of its technical difficulty, but also due to its importance to many commercial applications, such as automatic karoake and remixing. In this work, we propose a novel self-attention network to separate voice and accompaniment in music. First, a convolutional neural network (CNN) with densely-connected CNN blocks is built as our base network. We then insert self-attention subnets at different levels of the base CNN to make use of the long-term intra-dependency of music, i.e., repetition. Within self-attention subnets, repetitions of the same musical patterns inform reconstruction of other repetitions, for better source separation performance. Results show the proposed method leads to 19.5% relative improvement in vocals separation in terms of SDR. We compare our methods with state-of-the-art systems i.e. MMDenseNet and MMDenseLSTM.