Word has been making its way out from the technology community: The world changed this summer with the rollout of an artificial intelligence system known as GPT-3. Its ability to interact in English and generate coherent writing have been startling hardened experts, who speak of "GPT-3 shock." Where typical AI systems are trained for specific tasks--classifying images, playing Go--GPT-3 can handle tasks it was never specifically trained for. Research released by its maker, San Francisco-based OpenAI, has found that GPT-3 can...

Since OpenAI first described its new AI language-generating system called GPT-3 in May, hundreds of media outlets (including MIT Technology Review) have written about the system and its capabilities. Twitter has been abuzz about its power and potential. The New York Times published an op-ed about it. Later this year, OpenAI will begin charging companies for access to GPT-3, hoping that its system can soon power a wide variety of AI products and services. Is GPT-3 an important step toward artificial general intelligence--the kind that would allow a machine to reason broadly in a manner similar to humans without having to train for every specific task it encounters?

The above are excerpts from a long reply to a few questioning letters written by nine eminent philosophers from Massachusetts Institute of Technology, Harvard, Cambridge University and others. These letters asked questions like: Can artificial intelligence (AI) be truly conscious--and will machines ever be able to "understand"? How does technology interact with the social world, in all its messy, unjust complexity? How might AI and machine learning transform the distribution of power in society, our political discourse, our personal relationships, and our aesthetic experiences? The questions were addressed to the most recent arrival in the world of AI, called GPT-3.

Dialogue systems pretrained with large language models generate locally coherent responses, but lack the fine-grained control over responses necessary to achieve specific goals. A promising method to control response generation is exemplar-based generation, in which models edit exemplar responses that are retrieved from training data, or hand-written to strategically address discourse-level goals, to fit new dialogue contexts. But, current exemplar-based approaches often excessively copy words from the exemplar responses, leading to incoherent replies. We present an Exemplar-based Dialogue Generation model, EDGE, that uses the semantic frames present in exemplar responses to guide generation. We show that controlling dialogue generation based on the semantic frames of exemplars, rather than words in the exemplar itself, improves the coherence of generated responses, while preserving semantic meaning and conversation goals present in exemplar responses.

NLP has reached the'image net' moment. This means there is an increasing ability to crack complex language problems with the newer language models. The availability of large open source pre-trained language models combined with transfer learning techniques has made it possible for users to solve complex problems with ease. This includes language translation, text classification, question answering, language understanding, and language generation. The advancement in NLP technology has fuelled a so-called war to build the next bigger, better language model that can beat competition by its sheer size and complexity of tasks that can be performed.

Despite the buzz around GPT-3, it is, in and of itself, not AGI. However, language modelling has one crucial difference from Chess or Go or image classification. By harnessing the world model embedded in the language model, it may be possible to build a proto- AGI.

It's nice to see a concise implementation of GPT, in pytorch, as it is true Hugging Face's Transformer's is excellent, but it is quite difficult to trace. They are trying to build it out constantly with loads of features, so you get lost. His wiki states he works for OpenAI and Tesla is at least affiliated with Openai. Also it's very far from computer vision domain, so why spend the time on an open source implementation and make some guesses on GPT-2/GPT-3. His implementation is easy to follow, which is nice, most reimplementations I see have bugs or are unecessary complex.

A bot that writes letters on behalf of nature. Those are just some of the recent stories written about GPT-3, the latest contraption of artificial intelligence research lab OpenAI. GPT-3 is the largest language model ever made, and it has triggered many discussions over how AI will soon transform many industries. But what has been less discussed is how GPT-3 has transformed OpenAI itself. In the process of creating the most successful natural language processing system ever created, OpenAI has gradually morphed from a nonprofit AI lab to a company that sells AI services. And hanging in the balance is the very mission for which OpenAI was founded.

It's the Swiss Army Knife of AI from OpenAI, a San Francisco R&D shop set up to guide a path to safe artificial general intelligence and funded by Microsoft, Reid Hoffman, and Vinod Khosla. Generative Pre-trained Transformer 3 (GPT-3) is a deep learning language model that produces human-like text. The third-generation model "is the most powerful language model ever," MIT Technology Review. The API for GPT-3 represents a sector advance more than a breakthrough. It performs like a clever student trying to fake their way through a course.

The COVID-19 pandemic, anti-racism protests, and an impending global recession might have pushed global warming off the public agenda, but don't worry -- the climate crisis is looking worse than ever. A new art project aims to revive interest in the emergency by giving a voice to the environment. Letters from Nature uses AI to write epistles to world leaders warning of the dangers, on behalf of the world's glaciers, coral reefs, ice caps, and disappearing islands. To generate the letters, artist Jeroen van der Most and AI researcher Peter van der Putten fed a short prompt to GPT-3, OpenAI's unnervingly powerful text generator. The model then spat back a climate protest letter, addressed to one of the world's most powerful people, and signed by one of its most threatened entities.