Goto

Collaborating Authors

language model


Why You Should Do NLP Beyond English

#artificialintelligence

Natural language processing (NLP) research predominantly focuses on developing methods that work well for English despite the many positive benefits of working on other languages. These benefits range from an outsized societal impact to modelling a wealth of linguistic features to avoiding overfitting as well as interesting challenges for machine learning (ML). There are around 7,000 languages spoken around the world. The map above (see the interactive version at Langscape) gives an overview of languages spoken around the world, with each green circle representing a native language. Most of the world's languages are spoken in Asia, Africa, the Pacific region and the Americas.


GPT-3 Will Accelerate The Privatization of Internet Communities

#artificialintelligence

Two Thursdays ago, I was sitting alone in my SoMa loft, screaming at my computer. I was reading something that can only be described as magic. It was a post about how to run an effective board meeting. Part of what makes it so hard to build a strong board is a lack of focus and direction. When you go out and recruit board members, you have to be very intentional about your recruiting efforts and have a defined process.


GPT-3 101: a brief introduction

#artificialintelligence

Let's start with the basics. GPT-3 stands for Generative Pretrained Transformer version 3, and it is a sequence transduction model. Simply put, sequence transduction is a technique that transforms an input sequence to an output sequence. GPT-3 is a language model, which means that, using sequence transduction, it can predict the likelihood of an output sequence given an input sequence. This can be used, for instance to predict which word makes the most sense given a text sequence.


Tyler Cowen - Artificial Intelligence Is the Hope 2020 Needs

#artificialintelligence

This year is likely to be remembered for the Covid-19 pandemic and for a significant presidential election, but there is a new contender for the most spectacularly newsworthy happening of 2020: the unveiling of GPT-3. As a very rough description, think of GPT-3 as giving computers a facility with words that they have had with numbers for a long time, and with images since about 2012. The core of GPT-3, which is a creation of OpenAI, an artificial intelligence company based in San Francisco, is a general language model designed to perform autofill. It is trained on uncategorized internet writings, and basically guesses what text ought to come next from any starting point. That may sound unglamorous, but a language model built for guessing with 175 billion parameters -- 10 times more than previous competitors -- is surprisingly powerful.


Philosophers On GPT-3 (updated with replies by GPT-3) - Daily Nous

#artificialintelligence

Nine philosophers explore the various issues and questions raised by the newly released language model, GPT-3, in this edition of Philosophers On, guest edited by Annette Zimmermann. Introduction Annette Zimmermann, guest editor GPT-3, a powerful, 175 billion parameter language model developed recently by OpenAI, has been galvanizing public debate and controversy. As the MIT Technology Review puts it: “OpenAI’s new language generator GPT-3 is shockingly good—and completely mindless”. Parts of the technology community hope (and fear) that GPT-3 could brings us one step closer to the hypothetical future possibility of human-like, highly sophisticated artificial general intelligence (AGI). Meanwhile, others (including OpenAI’s own CEO) have critiqued claims about GPT-3’s ostensible proximity to AGI, arguing that they are vastly overstated. Why the hype? As is turns out, GPT-3 is unlike other natural language processing (NLP) systems, the latter of which often struggle with what comes comparatively easily to humans: performing entirely new language tasks based on a few simple instructions and examples. Instead, NLP systems usually have to be pre-trained on a large corpus of text, and then fine-tuned in order to successfully perform a specific task. GPT-3, by contrast, does not require fine tuning of this kind: it seems to be able to perform a whole range of tasks reasonably well, from producing fiction, poetry, and press releases to functioning code, and from music, jokes, and technical manuals, to “news articles which human evaluators have difficulty distinguishing from articles written by humans”. The Philosophers On series contains group posts on issues of current interest, with the aim being to show what the careful thinking characteristic of philosophers (and occasionally scholars in related fields) can bring to popular ongoing conversations. Contributors present not fully worked out position papers but rather brief thoughts that can serve as prompts for further reflection and discussion. The contributors to this installment of “Philosophers On” are Amanda Askell (Research Scientist, OpenAI), David Chalmers (Professor of Philosophy, New York University), Justin Khoo (Associate Professor of Philosophy, Massachusetts Institute of Technology), Carlos Montemayor (Professor of Philosophy, San Francisco State University), C. Thi Nguyen (Associate Professor of Philosophy, University of Utah), Regina Rini (Canada Research Chair in Philosophy of Moral and Social Cognition, York University), Henry Shevlin (Research Associate, Leverhulme Centre for..


OpenAI's latest breakthrough is astonishingly powerful, but still fighting its flaws

#artificialintelligence

The most exciting new arrival in the world of AI looks, on the surface, disarmingly simple. It's not some subtle game-playing program that can outthink humanity's finest or a mechanically advanced robot that backflips like an Olympian. You start typing and it predicts what comes next. But while this sounds simple, it's an invention that could end up defining the decade to come. The program itself is called GPT-3 and it's the work of San Francisco-based AI lab OpenAI, an outfit that was founded with the ambitious (some say delusional) goal of steering the development of artificial general intelligence or AGI: computer programs that possess all the depth, variety, and flexibility of the human mind. For some observers, GPT-3 -- while very definitely not AGI -- could well be the first step toward creating this sort of intelligence.


OpenAI's new GPT-3 language explained in under 3 minutes

#artificialintelligence

So, you've seen some amazing GPT-3 demos on Twitter (if not, where have you been?). This mega machine learning model, created by OpenAI, can write it's own op-eds, poems, articles, and even working code: With GPT-3, I built a layout generator where you just describe any layout you want, and it generates the JSX code for you. GPT3()… the spreadsheet function to rule them all. Impressed with how well it pattern matches from a few examples. The same function looked up state populations, peoples' twitter usernames and employers, and did some math.


The (Un)ethical Story of GPT-3: OpenAI's Million Dollar Model

#artificialintelligence

Back on October 12, 2019, the world witnessed a previously unimaginable accomplishment- the first sub-two-hour marathon was run in an incredible time of 1:59:40 by Kenyan native Eliud Kipchoge. He would later say in regards to the amazing achievement that he "expected more people all over the world to run under 2 hours after today" [1]. While Kipchoge set new records in long distance running, across the world a team of natural language processing (NLP) experts at OpenAI, the Elon Musk-backed AI firm, published a new transformer-based language model with 1.5 billion parameters that achieved previously unthinkable performance in nearly every language task it faced [2]. The main takeaway from the paper by many experts was that bigger is better-the intelligence of transformer models can dramatically increase with the scale of parameters. In March of 2020, this theory gained support with OpenAI's release of version three of the model or GPT-3 which encapsulates a staggering 175 billion parameters and achieved even more remarkable performance than version 2, despite sharing, quite literally, the same architecture [3].


GPT-3 101: a brief introduction

#artificialintelligence

Let's start with the basics. GPT-3 stands for Generative Pretrained Transformer version 3, and it is a sequence transduction model. Simply put, sequence transduction is a technique that transforms an input sequence to an output sequence. GPT-3 is a language model, which means that, using sequence transduction, it can predict the likelihood of an output sequence given an input sequence. This can be used, for instance to predict which word makes the most sense given a text sequence.


GPT-3 is the future. But what can NLP do in the present?

#artificialintelligence

A lot of ink has been spilled (or pixels illuminated) about the wonders of GPT-3, OpenAI's latest and greatest language model. A team of more than 30 OpenAI researchers have released a paper about GPT-3, a language model capable of achieving state-of-the-art results on a set of benchmark and unique natural language processing tasks that range from language translation to generating news articles to answering SAT questions. But like most examples spat out by language models, almost all of these were hand-selected by humans after many runs. Because not-so-good results just wouldn't make the news. Even bearing that in mind, I'm still blown away by what I've seen of GPT-3.