I recently started a new newsletter focus on AI education. TheSequence is a no-BS( meaning no hype, no news etc) AI-focused newsletter that takes 5 minutes to read. The goal is to keep you up to date with machine learning projects, research papers and concepts. Safety is one of the emerging concerns in deep learning systems. In the context of deep learning systems, safety is related to building agents that respect safety dynamics in a given environment.

Reinforcement Learning (RL) possesses immense potential and is doubtless one of the most dynamic and stimulating fields of research in Artificial Intelligence. RL is considered as a game-changer in Data Science, particularly after observing the winnings of AI agents AlphaGo Zero and OpenAI Five against top human champions. However, RL is not restricted to games. The progress in Reinforcement Learning, especially during the last few years, has been sensational. RL is everywhere now, ranging from resource management to chemistry, from healthcare to finance, and from Recommender Systems to more advanced applications in stock prediction.

Keeping up with the trend of many recent years, Deep Learning in 2020 continued to be one of the fastest-growing fields, darting straight ahead into the Future of Work. The developments were manifold and on multiple fronts. OpenAI, the AI Research organization, declared PyTorch as its new standard Deep Learning framework. PyTorch will increase its research productivity at scale on GPUs. With PyTorch backing it, OpenAI cut down its generative modeling iteration time from weeks to days. Megvii Technology, a China-based startup, said that it would make its Deep Learning framework open-source.

The cloud giant is introducing the Gaudi instances at an opportune time. AI models are getting more complex, partially because enterprise machine learning initiatives are maturing and partially because research conducted by the likes of OpenAI is facilitating bigger neural network architectures. As neural networks grow in complexity, the amount of computing power necessary to train them is increasing and fueling demand for more efficient training infrastructure.

Deep generative modeling has seen impressive advances in recent years, to the point where it is now commonplace to see simulated samples (e.g., images) that closely resemble real-world data. However, generation quality is generally inconsistent for any given model and can vary dramatically between samples. We introduce Discriminator Gradient f low (DGf low), a new technique that improves generated samples via the gradient flow of entropy-regularized f-divergences between the real and the generated data distributions. The gradient flow takes the form of a nonlinear Fokker-Plank equation, which can be easily simulated by sampling from the equivalent McKean-Vlasov process. By refining inferior samples, our technique avoids wasteful sample rejection used by previous methods (DRS & MH-GAN). Compared to existing works that focus on specific GAN variants, we show our refinement approach can be applied to GANs with vector-valued critics and even other deep generative models such as VAEs and Normalizing Flows. Empirical results on multiple synthetic, image, and text datasets demonstrate that DGf low leads to significant improvement in the quality of generated samples for a variety of generative models, outperforming the state-of-the-art Discriminator Optimal Transport (DOT) and Discriminator Driven Latent Sampling (DDLS) methods. Deep generative models (DGMs) have excelled at numerous tasks, from generating realistic images (Brock et al., 2019) to learning policies in reinforcement learning (Ho & Ermon, 2016).

The company OpenAI has developed an extremely powerful machine-learning system that can rapidly generate text with minimal human input. The system is known as GPT-3 and it does everything from crafting an email to writing advanced fiction. However, the FT's innovation editor, John Thornhill, explains, there are barriers and even a dark side to this tool. A transcript for this podcast is currently unavailable, view our accessibility guide.

The Microsoft license is exclusive however, meaning Microsoft's cloud computing competitors cannot access it in the same way. The agreement was seen as important to helping OpenAI with the expense of getting GPT-3 up and running and maintaining it, according to an account in TechTalks. These include an estimated $10 million in expenses to research GPT-3 and train the model, tens of thousands of dollars in monthly cloud computing and electricity costs to run the models, an estimated one million dollars annually to retrain the model to prevent decay, and additional costs of customer support, marketing, IT, legal and other requirements to put a software product on the market.

Despite the challenges of 2020, the AI research community produced a number of meaningful technical breakthroughs. GPT-3 by OpenAI may be the most famous, but there are definitely many other research papers worth your attention. For example, teams from Google introduced a revolutionary chatbot, Meena, and EfficientDet object detectors in image recognition. Researchers from Yale introduced a novel AdaBelief optimizer that combines many benefits of existing optimization methods. OpenAI researchers demonstrated how deep reinforcement learning techniques can achieve superhuman performance in Dota 2. To help you catch up on essential reading, we've summarized 10 important machine learning research papers from 2020. These papers will give you a broad overview of AI research advancements this year.

Despite the challenges of 2020, the AI research community produced a number of meaningful technical breakthroughs. GPT-3 by OpenAI may be the most famous, but there are definitely many other research papers worth your attention. For example, teams from Google introduced a revolutionary chatbot, Meena, and EfficientDet object detectors in image recognition. Researchers from Yale introduced a novel AdaBelief optimizer that combines many benefits of existing optimization methods. OpenAI researchers demonstrated how deep reinforcement learning techniques can achieve superhuman performance in Dota 2. To help you catch up on essential reading, we've summarized 10 important machine learning research papers from 2020. These papers will give you a broad overview of AI research advancements this year. Of course, there are many more breakthrough papers worth reading as well.

Humans surpass the cognitive abilities of most other animals in our ability to "chunk" concepts into words, and then combine the words to combine the concepts. In this process, we make "infinite use of finite means", enabling us to learn new concepts quickly and nest concepts within each-other. While program induction and synthesis remain at the heart of foundational theories of artificial intelligence, only recently has the community moved forward in attempting to use program learning as a benchmark task itself. The cognitive science community has thus often assumed that if the brain has simulation and reasoning capabilities equivalent to a universal computer, then it must employ a serialized, symbolic representation. Here we confront that assumption, and provide a counterexample in which compositionality is expressed via network structure: the free category prior over programs. We show how our formalism allows neural networks to serve as primitives in probabilistic programs. We learn both program structure and model parameters end-to-end.