The bot was trained on three years' worth of posts from 4chan, the repulsive cousin of Reddit. Kilchner fed the bot threads from the Politically Incorrect /pol/ board, a 4chan message board notorious for racist, xenophobic, and hateful content. The bot sparked a heated debate on social media before it went offline. This is the worst AI ever! I trained a language model on 4chan's /pol/ board and the result is…. Watch here (warning: may be offensive):https://t.co/lihsaYAm7l pic.twitter.com/xs7rgtucQb

Large Language Models have taken the world by storm recently. The capabilities shown by these models, combined with the way it seems like they can do everything have gotten the AI community very excited (and some AGI doomers terrified, lol). As LLMs become more powerful, we will naturally see them serve as foundation tasks for all kinds of applications. The impact they will have can't be overstated. However, it's crucial to ensure that these models are safe and don't come with seriously problematic biases/cases.

Everybody knows that large language models like GPT-3 and LaMDA have made tremendous strides, at least in some respects, and powered past many benchmarks, and Cosmo recently described DALL-E but most in the field also agree that something is still missing. A growing body of evidence shows that state-of-the-art models learn to exploit spurious statistical patterns in datasets... instead of learning meaning in the flexible and generalizable way that humans do." Since then, the results on benchmarks have gotten better, but there's still something missing. Reference: Words and sentence don't exist in isolation. Language is about a connection between words (or sentence) and the world; the sequences of words that large language models utter lack connection to the external world.

A transformer has strong language representation ability; a very large corpus contains rich language expressions (such unlabeled data can be easily obtained) and training large-scale deep learning models has become more efficient. Therefore, pre-trained language models can effectively represent a language's lexical, syntactic, and semantic features. Pre-trained language models, such as BERT and GPTs (GPT-1, GPT-2, and GPT-3), have become the core technologies of current NLP. Pre-trained language model applications have brought great success to NLP. "Fine-tuned" BERT has outperformed humans in terms of accuracy in language-understanding tasks, such as reading comprehension.8,17 "Fine-tuned" GPT-3 has also reached an astonishing level of fluency in text-generation tasks.3

We're making GitHub Copilot, an AI pair programmer that suggests code in your editor, generally available to all developers for $10 USD/month or $100 USD/year. It will also be free to use for verified students and maintainers of popular open source projects.

Is thinking of autonomous vehicles as AI systems rather than as robots the next step forward? A new wave of startups is trying techniques such as reinforcement learning to train AVs to drive safely. Generative Flow Networks may be the next major step in building better AI systems. The ethics of building AI bots that mimic real dead people seems like an academic question, until someone does it: using GPT-3, a developer created a bot based on his deceased fiancée. OpenAI objected, stating that building such a bot was a violation of its terms of service.

We are excited to bring Transform 2022 back in-person July 19 and virtually July 20 - 28. Join AI and data leaders for insightful talks and exciting networking opportunities. All hell broke loose in the AI world after The Washington Post reported last week that a Google engineer thought that LaMDA, one of the company's large language models (LLM), was sentient. The news was followed by a frenzy of articles, videos and social media debates over whether current AI systems understand the world as we do, whether AI systems can be conscious, what are the requirements for consciousness, etc. We are currently in a state where our large language models have become good enough to convince many people -- including engineers -- that they are on par with natural intelligence. At the same time, they are still bad enough to make dumb mistakes, as these experiments by computer scientist Ernest Davis show.

Human-machine collaboration is an interesting field in the development of creative systems. It would be ideal if the system could produce novel creative outputs, explain them and discuss them with their contributors. Researchers leverage the capabilities of transformer-based technologies to generate explanations of musical decisions using GPT-3, a state-of-the-art natural language model. It was fed with example pieces of the music with explanations provided by the author. Then, the capability to produce explanations itself was tested.

LaMDA has been in the news after a Google engineer claimed it was sentient because its answers allegedly hint that it understands what it is. The engineer also suggested that LaMDA communicates that it has fears, much like a human does. What is LaMDA, and why are some under the impression that it can achieve consciousness? LaMDA is a language model. Fundamentally, it's a mathematical function (or a statistical tool) that describes a possible outcome related to predicting what the next words are in a sequence.

This article is part of Demystifying AI, a series of posts that (try to) disambiguate the jargon and myths surrounding AI. The past week has seen a frenzy of articles, interviews, and other types of media coverage about Blake Lemoine, a Google engineer who told The Washington Post that LaMDA, a large language model created for conversations with users, is "sentient." After reading a dozen different takes on the topic, I have to say that the media has become (a bit) disillusioned with the hype surrounding current AI technology. A lot of the articles discussed why deep neural networks are not "sentient" or "conscious." This is an improvement in comparison to a few years ago, when news outlets were creating sensational stories about AI systems inventing their own language, taking over every job, and accelerating toward artificial general intelligence.