What happens when robots not only learn to write well, but the tech becomes easily accessible and cheap? As Hal Crawford explains, it'll likely be teachers who feel the effects first. There are two schools of thought when it comes to artificial intelligence: there are the people who have heard of the GPT-3 language model, and then there are those who have heard about it, gone to the OpenAI site, created a guest login and tried it out for themselves. The first group contains people who are wondering what the big deal is. The second group does not. I haven't heard of anyone who's actually used GPT-3 and doesn't think AI is going to change the world profoundly. Education in particular is going to feel its influence immediately.

"You can't go to a physics conference and say: I've got a great theory. It accounts for everything and is so simple it can be captured in two words: "Anything goes."" Every now and then engineers make an advance, and scientists and lay people begin to ponder the question of whether that advance might yield important insight into the human mind. Descartes wondered whether the mind might work on hydraulic principles; throughout the second half of the 20th century, many wondered whether the digital computer would offer a natural metaphor for the mind. The latest hypothesis to attract notice, both within the scientific community, and in the world at large, is the notion that a technology that is popular today, known as large language models, such as OpenAI's GPT-3, might offer important insight into the mechanics of the human mind. Enthusiasm for such models has grown rapidly; OpenAI's Chief Science Officer Ilya Sutskever recently suggested that such systems could conceivably be "slightly conscious".

Earlier this month, DeepMind presented a new "generalist" AI model called Gato. The model can play the video game Atari, caption images, chat, and stack blocks with a real robot arm, the Alphabet-owned AI lab announced. All in all, Gato can do hundreds of different tasks. But while Gato is undeniably fascinating, in the week since its release some researchers have got a bit carried away. One of DeepMind's top researchers and a coauthor of the Gato paper, Nando de Freitas, couldn't contain his excitement.

Does the breakthrough to general AI need more data and computing power above all else? Yann LeCun, Chief AI Scientist at Meta, comments on the recent debate about scaling sparked by Deepmind's Gato. The recent successes of large AI models such as OpenAI's DALL-E 2, Google's PaLM and Deepmind's Flamingo have sparked a debate about their significance for progress towards general AI. Deepmind's Gato has recently given a particular boost to the debate, which has been conducted publicly, especially on Twitter. Gato is a Transformer model trained with numerous data modalities, including images, text, proprioception or joint moments.

The company has raised $100 million in round C funding with the aim of becoming the "GitHub of machine learning". Inflection -- is an AI-first company aiming to redefine human-computer interaction. It is led by LinkedIn and DeepMind co-founders and was referenced in our Newsletter #68. The company has now raised $225 million in venture funding to use AI to help humans "talk" to computers. Unlearn -- aims to accelerate clinical trials by using AI, digital twins, and novel statistical methods to "enable smaller control groups while maintaining power and generating evidence suitable for supporting regulatory decisions".

Google is being sued over its use of confidential medical records belonging to 1.6 million individuals in the UK. The company's artificial intelligence arm, DeepMind, received the data in 2015 from the Royal Free NHS Trust in London for the purpose of testing a smartphone app called Streams. The claim is being brought by Andrew Prismall in a representative action in the High Court. It alleges that Google and DeepMind "obtained and used a substantial number of confidential medical records without patients' knowledge or consent". Why did Google get access to patient records?

Human-level artificial intelligence is close to finally being achieved, according to a lead researcher at Google's DeepMind AI division. Dr Nando de Freitas said "the game is over" in the decades-long quest to realise artificial general intelligence (AGI) after DeepMind unveiled an AI system capable of completing a wide range of complex tasks, from stacking blocks to writing poetry.

GPT-3 is a large-scale natural language model developed by OpenAI that can perform many different tasks, including topic classification. Although researchers claim that it requires only a small number of in-context examples to learn a task, in practice GPT-3 requires these training examples to be either of exceptional quality or a higher quantity than easily created by hand. To address this issue, this study teaches GPT-3 to classify whether a question is related to data science by augmenting a small training set with additional examples generated by GPT-3 itself. This study compares two classifiers: the GPT-3 Classification Endpoint with augmented examples, and the GPT-3 Completion Endpoint with an optimal training set chosen using a genetic algorithm. We find that while the augmented Completion Endpoint achieves upwards of 80 percent validation accuracy, using the augmented Classification Endpoint yields more consistent accuracy on unseen examples. In this way, giving large-scale machine learning models like GPT-3 the ability to propose their own additional training examples can result in improved classification performance.

Want to ensure your app developers can create secure and smooth login experiences for your customers? With Curity you can protect user identities, secure apps and websites, and manage API access. Welcome to the InfoQ podcast. My name is Roland Meertens and today, I am interviewing Cassie Breviu. She is a senior program manager at Microsoft and hosted the innovations in machine learning systems track at QCon London. I am actually speaking to her in person at the venue of QCon London Conference. In this interview, I will talk with her on how she got started with AI and what machine learning tools can accelerate your work when deploying models on a wide range of devices. We will also talk about GitHub Copilot and how AI can help you be a better programmer. If you want to see her talk on how to operationalize transformer models on the edge, at the moment of recording this, you can still register for the QCon Plus Conference or see if the recording is already uploaded on infoq.com. Welcome, Cassie to QCon London. I'm very glad to see you here. I hope you're happy to be at this conference. I heard that you actually got into AI by being at the conference. I am thoroughly enjoying this conference. It's really put together really well and I really enjoy it. So what happened was I was at a developer conference. I was a full stack C# engineer and I'd always been really interested in AI and machine learning, but it always seemed scary and out of reach. I had even tried to read some books on it and I thought, "Well, this might be just too much for me or too complicated or I just can't do this." So I went to this talk by Jennifer Marsman and she did this amazing talk on, Would You Survive the Titanic Sinking? She used this product that's called Azure Machine Learning Designer.

Since their introduction in 2017, transformers have become the go-to machine learning architecture for natural language processing (NLP) and computer vision. Although they have achieved state-of-the-art performance in these fields, the theoretical framework underlying transformers remains relatively underexplored. In the new paper A Probabilistic Interpretation of Transformers, ML Collective researcher Alexander Shim provides a probabilistic explanation of transformers' exponential dot product attention and contrastive learning based on distributions of the exponential family. An oft-proposed explanation for transformers' power and performance is their attention mechanisms' superior ability to model dependencies in long input sequences. But this doesn't directly address how and why transformer architecture choices such as exponential dot product attention outperform the alternatives.