Effectively leveraging large, previously collected datasets in reinforcement learning (RL) is a key challenge for large-scale real-world applications. Offline RL algorithms promise to learn effective policies from previously-collected, static datasets without further interaction. However, in practice, offline RL presents a major challenge, and standard off-policy RL methods can fail due to overestimation of values induced by the distributional shift between the dataset and the learned policy, especially when training on complex and multi-modal data distributions. In this paper, we propose conservative Q-learning (CQL), which aims to address these limitations by learning a conservative Q-function such that the expected value of a policy under this Q-function lower-bounds its true value. We theoretically show that CQL produces a lower bound on the value of the current policy and that it can be incorporated into a policy learning procedure with theoretical improvement guarantees. In practice, CQL augments the standard Bellman error objective with a simple Q-value regularizer which is straightforward to implement on top of existing deep Q-learning and actor-critic implementations. On both discrete and continuous control domains, we show that CQL substantially outperforms existing offline RL methods, often learning policies that attain 2-5 times higher final return, especially when learning from complex and multi-modal data distributions.

This is one of the many articles published back in 2017 that claimed that a catastrophic AI-related event happened within the gates of Facebook: two newborn chatbots, communicating with each other, started developing their own language. This is how the event was reported: the bots began interacting with each other and started to create their own way of communicating deviating from the human language. In addition, the Facebook engineer had to stop the experiment'to prevent the worst'. As you can see from the screenshot, this is the conversation that took place between the two AI chatbots. Now, what is really going on?

Originally developed by me (Nicklas Hansen), Peter E. Christensen and Alexander R. Johansen as educational material for the graduate deep learning course at the Technical University of Denmark (DTU). You can access the full course material here. Inspired by the great Andrej Karpathy. In this lab we will introduce different ways of learning from sequential data. As an example, we will train a neural network to do language modelling, i.e. predict the next token in a sentence.

Any researcher who's focused on applying machine learning to real-world problems has likely received a response like this one: "The authors present a solution for an original and highly motivating problem, but it is an application and the significance seems limited for the machine-learning community." These words are straight from a review I received for a paper I submitted to the NeurIPS (Neural Information Processing Systems) conference, a top venue for machine-learning research. I've seen the refrain time and again in reviews of papers where my coauthors and I presented a method motivated by an application, and I've heard similar stories from countless others. This makes me wonder: If the community feels that aiming to solve high-impact real-world problems with machine learning is of limited significance, then what are we trying to achieve? The goal of artificial intelligence (pdf) is to push forward the frontier of machine intelligence. In the field of machine learning, a novel development usually means a new algorithm or procedure, or--in the case of deep learning--a new network architecture.

Despite the buzz around GPT-3, it is, in and of itself, not AGI. However, language modelling has one crucial difference from Chess or Go or image classification. By harnessing the world model embedded in the language model, it may be possible to build a proto- AGI.

Fine-tuning deep learning models just got easier with the black box adversarial reprogramming (BAR) technique. When data scientists mention AI and machine learning models, the hot topic of discussion always revolves around not having enough training samples to fine-tune the deep learning models. Consequently, they rely on transfer learning to subsequently fine-tune pre-train deep learning models to increase a model's accuracy. To make data scientists work a lot easier, at the International Conference on Machine Learning (ICML) scientists at IBM research and Taiwan's National Tsing Hua University unrevealed the Black Box Adversarial Reprogramming (BAR) touted as an alternative repurposing technique which turns the weakness of deep neural networks into a strength. Explaining BAR, the research paper presented at the paper read, "Black Box Adversarial Reprogramming repurposes a well-trained black-box ML model for solving different ML tasks, especially in the scenario with scarce data and constrained resources. The rationale lies in exploiting high-performance but unknown ML models to gain learning capability for transfer learning. Using zeroth order optimization and multi-label mapping techniques, BAR can reprogram a black-box ML model solely based on its input-output responses without knowing the model architecture or changing any parameter. More importantly, in the limited medical data setting, on autism spectrum disorder classification, diabetic retinopathy detection, and melanoma detection tasks, BAR outperforms state-of-the-art methods and yields comparable performance to the vanilla adversarial reprogramming method requiring complete knowledge of the target ML model. BAR also outperforms baseline transfer learning approaches by a significant margin, demonstrating cost-effective means and new insights for transfer learning".

Deep learning technology has rattled the global financial industry in both positive and negative ways. On the one hand, deep learning technology has considerably improved market efficiency. Tomiwa, a big data author and expert, claims to have beaten the stock market average over the past ten years with a program that he developed with Python. The same kind of program could be used by Forex or derivative traders. One of the biggest downsides, though, is that it has giving larger institutional traders with deep pockets an even stronger advantage.

It's nice to see a concise implementation of GPT, in pytorch, as it is true Hugging Face's Transformer's is excellent, but it is quite difficult to trace. They are trying to build it out constantly with loads of features, so you get lost. His wiki states he works for OpenAI and Tesla is at least affiliated with Openai. Also it's very far from computer vision domain, so why spend the time on an open source implementation and make some guesses on GPT-2/GPT-3. His implementation is easy to follow, which is nice, most reimplementations I see have bugs or are unecessary complex.

Text classification is one of the important applications of NLP. Applications such as Sentiment Analysis and Identifying spam, bots, and offensive comments come under Text Classification. Until now, the approaches used for solving these problems included building Machine Learning or Deep Learning models from scratch, training them on your text data, and fine-tuning it with hyperparameters. Even though such models give decent results for applications like classifying whether a movie review is positive or negative, they may perform terribly if things become more ambiguous because most of the time there's just not enough amount of labeled data to learn from. Isn't the Imagenet using the same approach to classify the images?

Image credit: DepositphotosThis article is part of Demystifying AI, a series of posts that (try to) disambiguate the jargon and myths surrounding AI. Passport checks at airports and border gates present a special challenge: How do you tell if the person standing in front of you is the same person whose picture is in the passport? Border and customs officers solve this problem using the complex mechanisms ingrained in the human visual system through billions of years of evolution. It's not a perfect process, but it works well most of the time. In the realm of artificial intelligence, this is called the "one-shot learning" challenge. In a more abstract way, can you develop a computer vision system that can look at two images it has never seen before and say whether they represent the same object?