OpenAI's paper describes many incredible tasks that GPT-3 can accomplish. For instance, given some text input, it can predict what should come next alarmingly well. In one study, test subjects were asked to differentiate between 500 word long articles written by humans or articles written by GTP-3. When GPT-3 is used with 175 billion parameters, they were only able to correctly identify text from other humans at a rate of 52%. That implies GPT-3 at maximum capacity can nearly replicate human written articles!

GPT-3 requires significant expertise and is too expensive for individuals and small businesses to run. This makes it hard for anyone except larger companies to benefit from the underlying technology. OpenAI gave this as one of 3 reasons why the model is only available via API and was not open-sourced. At 175 billion parameters, GPT-3 dwarfs GPT-2 which had "only" 1.5 billion parameters. Given the death of Moores Law, we won't see such models running on personal computers in the near future.

Advances in artificial intelligence are often framed as though they'll upend human society overnight, and for the last month technologists have had a new darling to obsess over. It's a piece of software called GPT-3, which can write essays, songs and computer code in response to requests that humans type into its interface in plain text. The program comes from OpenAI, a top artificial intelligence research organization. OpenAI recently began giving technologists access to it in a private testing period, and they are in love. GPT-3 can build a website layout based on a text-based description, or write a Harry Potter story in the style of a hard-boiled detective novel.

I typically think of the dog days of summer as a time when news slows down. It's typically when a lot of people take time off work, and the lull leads local news stations to cover inconsequential things like cat shows or a little baby squirrel on a little baby Jet Ski. But these are not typical times. Fallout surrounding issues of bias and discrimination continues at Facebook, as multiple news outlets reported that Instagram's content moderation algorithm was 50% more likely to flag and disable the accounts of Black users than White users. Facebook and Instagram are now creating teams to examine how algorithms impact the experiences of Black, Latinx, and other specific groups of users.

Recent progress in scientific machine learning (SciML) has opened up the possibility of training novel neural network architectures that solve complex partial differential equations (PDEs). Several (nearly data free) approaches have been recently reported that successfully solve PDEs, with examples including deep feed forward networks, generative networks, and deep encoder-decoder networks. However, practical adoption of these approaches is limited by the difficulty in training these models, especially to make predictions at large output resolutions ($\geq 1024 \times 1024$). Here we report on a software framework for data parallel distributed deep learning that resolves the twin challenges of training these large SciML models - training in reasonable time as well as distributing the storage requirements. Our framework provides several out of the box functionality including (a) loss integrity independent of number of processes, (b) synchronized batch normalization, and (c) distributed higher-order optimization methods. We show excellent scalability of this framework on both cloud as well as HPC clusters, and report on the interplay between bandwidth, network topology and bare metal vs cloud. We deploy this approach to train generative models of sizes hitherto not possible, showing that neural PDE solvers can be viably trained for practical applications. We also demonstrate that distributed higher-order optimization methods are $2-3\times$ faster than stochastic gradient-based methods and provide minimal convergence drift with higher batch-size.

With the recent tweet of Sharif Shameem of debuild, where he showed off a unique feature of OpenAI's GPT-3 which can generate codes, it has raised an interesting question in the programming scene. Whether this new language model released by OpenAI will kill coding or will create more productive programmers in the future. Being trained on billions of words from the internet has made GPT-3 capable of creating codes in CSS, JSX, Python etc. What's more, GPT-3 doesn't need to be trained at all for specific tasks, working on zero-shot learning, which makes it the largest model in the AI world with a strong performance on many NLP datasets allowing it to perform a range of tasks. Explaining the process, he said, "all I had to do is to re-write my two initial samples, and the GPT generated outputs in plain HTML/CSS." With GPT-3, I built a layout generator where you just describe any layout you want, and it generates the JSX code for you.

A new general language machine learning model is pushing the boundaries of what AI can do. Why it matters: OpenAI's GPT-3 system can reasonably make sense of and write human language. It's still a long way from genuine artificial intelligence, but it may be looked back on as the iPhone of AI, opening the door to countless commercial applications -- both benign and potentially dangerous. Driving the news: After announcing GPT-3 in a paper in May, OpenAI recently began offering a select group of people access to the system's API to help the nonprofit explore the AI's full capabilities. How it works: GPT-3 works the same way as predecessors like OpenAI's GPT-2 and Google's BERT -- analyzing huge swathes of the written internet and using that information to predict which words tend to follow after each other.

"I share my early experiments with OpenAI's new language prediction model (GPT-3) beta. I explain why I think GPT-3 has disruptive potential comparable to that of blockchain technology." Araoz put GPT-3 to the test in several other ways, using it to make complex texts more understandable, to write poetry in the style of Borges in the Spanish language and write music in ABC notation.

Developer Manuel Araoz has played a practical joke online to demonstrate the potential of artificial intelligence bots -- by having a bot write an article about itself. According to a July 18 post on Araoz's blog, AI development company OpenAI released GPT-3, the third generation of its language prediction model capable of creating "random-ish sentences of approximately the same length and grammatical structure as those in a given body of text." The blog entry provides practical information regarding how the technology could be used to impersonate well-known figures by simulating their writing styles -- for example, Araoz used it to create a fake interview with Albert Einstein. He predicted that the GPT-3 could potentially replace journalists, political speech writers, and advertising copywriters. The bot's predicted sentences were used for posts on the bitcointalk.org "There are lots of posts for GPT-3 to study and learn from.

All said, I'm extremely excited to see which new technologies are built on GPT-3 and how OpenAI continues to improve on its model. Increased attention and funding in NLP and GPT-3 might be enough to ward off fears from many critics that an AI winter might be coming (myself included). Despite the shortfalls of the model, I am hoping that everyone can be optimistic about a future where humans and machines will communicate with each other in a unified language and the ability to create tools using technology will be accessible to billions of more people.