Nine philosophers explore the various issues and questions raised by the newly released language model, GPT-3, in this edition of Philosophers On, guest edited by Annette Zimmermann. Introduction Annette Zimmermann, guest editor GPT-3, a powerful, 175 billion parameter language model developed recently by OpenAI, has been galvanizing public debate and controversy. As the MIT Technology Review puts it: “OpenAI’s new language generator GPT-3 is shockingly good—and completely mindless”. Parts of the technology community hope (and fear) that GPT-3 could brings us one step closer to the hypothetical future possibility of human-like, highly sophisticated artificial general intelligence (AGI). Meanwhile, others (including OpenAI’s own CEO) have critiqued claims about GPT-3’s ostensible proximity to AGI, arguing that they are vastly overstated. Why the hype? As is turns out, GPT-3 is unlike other natural language processing (NLP) systems, the latter of which often struggle with what comes comparatively easily to humans: performing entirely new language tasks based on a few simple instructions and examples. Instead, NLP systems usually have to be pre-trained on a large corpus of text, and then fine-tuned in order to successfully perform a specific task. GPT-3, by contrast, does not require fine tuning of this kind: it seems to be able to perform a whole range of tasks reasonably well, from producing fiction, poetry, and press releases to functioning code, and from music, jokes, and technical manuals, to “news articles which human evaluators have difficulty distinguishing from articles written by humans”. The Philosophers On series contains group posts on issues of current interest, with the aim being to show what the careful thinking characteristic of philosophers (and occasionally scholars in related fields) can bring to popular ongoing conversations. Contributors present not fully worked out position papers but rather brief thoughts that can serve as prompts for further reflection and discussion. The contributors to this installment of “Philosophers On” are Amanda Askell (Research Scientist, OpenAI), David Chalmers (Professor of Philosophy, New York University), Justin Khoo (Associate Professor of Philosophy, Massachusetts Institute of Technology), Carlos Montemayor (Professor of Philosophy, San Francisco State University), C. Thi Nguyen (Associate Professor of Philosophy, University of Utah), Regina Rini (Canada Research Chair in Philosophy of Moral and Social Cognition, York University), Henry Shevlin (Research Associate, Leverhulme Centre for..

The most exciting new arrival in the world of AI looks, on the surface, disarmingly simple. It's not some subtle game-playing program that can outthink humanity's finest or a mechanically advanced robot that backflips like an Olympian. You start typing and it predicts what comes next. But while this sounds simple, it's an invention that could end up defining the decade to come. The program itself is called GPT-3 and it's the work of San Francisco-based AI lab OpenAI, an outfit that was founded with the ambitious (some say delusional) goal of steering the development of artificial general intelligence or AGI: computer programs that possess all the depth, variety, and flexibility of the human mind. For some observers, GPT-3 -- while very definitely not AGI -- could well be the first step toward creating this sort of intelligence.

So, you've seen some amazing GPT-3 demos on Twitter (if not, where have you been?). This mega machine learning model, created by OpenAI, can write it's own op-eds, poems, articles, and even working code: With GPT-3, I built a layout generator where you just describe any layout you want, and it generates the JSX code for you. GPT3()… the spreadsheet function to rule them all. Impressed with how well it pattern matches from a few examples. The same function looked up state populations, peoples' twitter usernames and employers, and did some math.

Back on October 12, 2019, the world witnessed a previously unimaginable accomplishment- the first sub-two-hour marathon was run in an incredible time of 1:59:40 by Kenyan native Eliud Kipchoge. He would later say in regards to the amazing achievement that he "expected more people all over the world to run under 2 hours after today" [1]. While Kipchoge set new records in long distance running, across the world a team of natural language processing (NLP) experts at OpenAI, the Elon Musk-backed AI firm, published a new transformer-based language model with 1.5 billion parameters that achieved previously unthinkable performance in nearly every language task it faced [2]. The main takeaway from the paper by many experts was that bigger is better-the intelligence of transformer models can dramatically increase with the scale of parameters. In March of 2020, this theory gained support with OpenAI's release of version three of the model or GPT-3 which encapsulates a staggering 175 billion parameters and achieved even more remarkable performance than version 2, despite sharing, quite literally, the same architecture [3].

Recent progress in scientific machine learning (SciML) has opened up the possibility of training novel neural network architectures that solve complex partial differential equations (PDEs). Several (nearly data free) approaches have been recently reported that successfully solve PDEs, with examples including deep feed forward networks, generative networks, and deep encoder-decoder networks. However, practical adoption of these approaches is limited by the difficulty in training these models, especially to make predictions at large output resolutions ( 1024 1024). Here we report on a software framework for data parallel distributed deep learning that resolves the twin challenges of training these large SciML models - training in reasonable time as well as distributing the storage requirements. Our framework provides several out of the box functionality including (a) loss integrity independent of number of processes, (b) synchronized batch normalization, and (c) distributed higher-order optimization methods.

Social media is awash with people talking about a new piece of software called GPT-3, which has been developed by OpenAI, an Elon Musk-backed artificial intelligence lab in San Francisco. GPT-3 (Generative Pre-training) is a language-generation tool capable of producing human-like text on demand. The software learned how to produce text by analyzing vast quantities on the internet and observing which letters and words tend to follow one another. OpenAI started releasing it to a select few people last week who had requested access to a private early version, and many of them have been blown away. "It's far more coherent than any AI language system I've ever tried," wrote entrepreneur Arram Sabeti in a blog post after testing. "All you have to do is write a prompt and it'll add text it thinks would plausibly follow.

Last spring, artificial intelligence research institute OpenAI said it had made software so good at generating text--including fake news articles--that it was too dangerous to release. That line in the sand was soon erased when two recent master's grads recreated the software and OpenAI released the original, saying awareness of the risks had grown and it hadn't seen evidence of misuse. Now the lab is back with a more powerful text generator and a new pitch: Pay us to put it to work in your business. Thursday, OpenAI launched a cloud service that a handful of companies are already using to improve search or provide feedback on answers to math problems. OpenAI was founded as a nonprofit in 2015 by Elon Musk and other Silicon Valley notables to ensure that future superhuman AI was a benign force.

A lot of ink has been spilled (or pixels illuminated) about the wonders of GPT-3, OpenAI's latest and greatest language model. A team of more than 30 OpenAI researchers have released a paper about GPT-3, a language model capable of achieving state-of-the-art results on a set of benchmark and unique natural language processing tasks that range from language translation to generating news articles to answering SAT questions. But like most examples spat out by language models, almost all of these were hand-selected by humans after many runs. Because not-so-good results just wouldn't make the news. Even bearing that in mind, I'm still blown away by what I've seen of GPT-3.

Did you know AI can now produce poetry and write fiction? That is not all it can also generate CODE ! Isn’t that amazing ! OpenAI’s newest AI language model – GPT-3, is trending all over the internet! Source: Unite.AI Background Elon Musk and Sam Altman started OpenAI in 2015 to advance the state of the art of AI and to ensure AI was used for the human good. OpenAI recently released the third version Generative Pre-training Transformer (GPT) – GPT-3. They first described GPT-3 through a research paper published in May but last week rolled out a beta version access to a select set of people and its capabilities are mind blowing! What is GPT-3 and why is everyone talking about it? Here are some quick pointers to help you understand what GPT-3 is, and why it is a step towards an AI revolution. Third version Generative Pre-training Transformer (GPT) A natural language generator (NLG) capable of producing human-like text on demand State-of-the-art language model made up of 175 billion parameters ~10X larger than Microsoft’s Turing NLG that has 17 billion parameters >100X larger than its own predecessor GPT-2 (released last year) trained on 1.5 billion parameters Does not require large custom, task specific datasets (which are usually difficult to get) Does not even require task specific model architectures What all can GPT-3 do for you? Here are a few things that GPT-3 can do for you and as more developers and experts experiment with it, more use cases will surface in the times to come. Answer questions with common sense (that doesn’t seem very difficult?) Write creative fiction – poetry, essays, stories Write news (Could that be a problem?) Solve arithmetic problems Generate Functioning Code (GPT-3 can CODE!) Design – developers demonstrated it with a Figma plugin (That’s truly creative!) This is just an illustrative list of what all GPT-3 can do! As more and more developers/experts experiment with the beta access, more innovations are bound to surface. Possible flaws and concerns pertaining to GPT-3? Although GPT-3 has proven brilliance in many ways in its current state but it also has certain flaws and concerns that it raises: Lacks an overarching, long-term sense of meaning and purpose As it generates its output word-by-word, based on the immediately surrounding text. It can struggle to maintain a coherent narrative or deliver a meaningful message over more than a few paragraphs is what experts say after initial experimentation. Possibility of being prone to certain biases Developers noticed that GPT-3 is prone to shoot out racist and sexist language, even when the prompt is something harmless. As GPT-3 is trained on internet scale data, these biases arise from biases in that training data reflecting possible societal views and opinions. Chances of misuse once released publicly Authors acknowledge that people can misuse it in several ways because of its ability to create text that is indistinguishable from that of written by humans. It can lead to creation of including generating misinformation and spam, phishing, and even fake academic essays. Possible impact on jobs As GPT-3 can perform human-like tasks across multiple domains, it could possibly have an impact on jobs that it can do like writers, coders, journalists, etc. These are just possible concerns and basis point 1 highlighted above many believe that instead of replacing humans, GPT-3 could become the perfect assistant for humans in these professions. Conclusion and what lies ahead? GPT-3 can execute plethora of NLP- based tasks, without fine-tuning for a specific task. Experts are even saying that this could be a step towards Artificial General Intelligence (AGI) – Read my article on Demystifying AI to know more! It is capable of performing machine translation, answering to questions, scripting poems and stories, elementary mathematics and can even generate code. OpenAI wants developers to help it explore what GPT-3 can do and with the beta release of its API has attracted plethora of experts and developers to experiment with the capabilities of GPT-3. It is only a matter of time before more exciting innovations surface in the developers’ community! Watch out for more interesting articles on AI! Feel free to share your thoughts. References [1] https://www.forbes.com/sites/robtoews/2020/07/19/gpt-3-is-amazingand-overhyped/#7c6166501b1c [2] https://analyticsindiamag.com/open-ai-gpt-3-code-generator-app-building/ [3] https://singularityhub.com/2020/06/18/openais-new-text-generator-writes-even-more-like-a-human/ [4] https://towardsdatascience.com/gpt-3-for-the-people-2cdd003d9a89 [5] https://www.technologyreview.com/2020/07/20/1005454/openai-machine-learning-language-generator-gpt-3-nlp/ [6] https://www.analyticssteps.com/blogs/what-openai-gpt-3 [7] https://www.cnbc.com/2020/07/23/openai-gpt3-explainer.html [8] https://analyticsindiamag.com/how-openais-gpt-3-can-be-alarming-for-the-society/ [9] https://www.forbes.com/sites/robtoews/2020/07/19/gpt-3-is-amazingand-overhyped/#7c6166501b1c [10] https://www.independent.co.uk/life-style/gadgets-and-tech/news/gpt3-ai-tool-designs-websites-medicine-a9627966.html [11] https://medium.com/fair-bytes/how-biased-is-gpt-3-5b2b91f1177 [12] https://towardsdatascience.com/gpt-3-demos-use-cases-implications-77f86e540dc1

The world has a new AI toy, and it's called GPT-3. The latest iteration of OpenAI's text generating model has left many starstruck by its abilities – although its hype may be too much. GPT-3 is a machine learning system that has been fed 45TB of text data, an unprecedented amount. All that training allows it to generate sorts of written content: stories, code, legal jargon, all based on just a few input words or sentences. And the beta test has already produced some jaw-dropping results.