Goto

Collaborating Authors

AI Compute Symposium Charts Path from Emerging to Pervasive AI

#artificialintelligence

Together with the IEEE Circuits and Systems Society and Electron Device Society, IBM Research organized the 2nd AI Compute Symposium at the IBM T.J. Watson Research Center THINKLab in Yorktown Heights, N.Y., on Oct 17. More than 200 distinguished academics, renowned thinkers, students, and innovators from across industry and academia assembled for the one-day symposium, which showcased leadership and advancement in research addressing AI compute from pervasive to general AI. The free event featured three keynotes, three invited talks, a student poster session, and a panel discussion. The keynoters were Dr. Luis Lastras, a researcher with IBM; Professor Wen-mei Hwu of the University of Illinois at Urbana-Champaign (UIUC); and Harvard University/Samsung Fellow Donhee Ham. Lastras provided an exciting overview of research projects from IBM related to natural language processing and its evolution.


Green AI

Communications of the ACM

Since 2012, the field of artificial intelligence (AI) has reported remarkable progress on a broad range of capabilities including object recognition, game playing, speech recognition, and machine translation.43 Much of this progress has been achieved by increasingly large and computationally intensive deep learning models.a Figure 1, reproduced from Amodei et al.,2 plots training cost increase over time for state-of-the-art deep learning models starting with AlexNet in 201224 to AlphaZero in 2017.45 The chart shows an overall increase of 300,000x, with training cost doubling every few months. An important paper47 has estimated the carbon footprint of several NLP models and argued this trend is both environmentally unfriendly and prohibitively expensive, raising barriers to participation in NLP research. We refer to such work as Red AI. The amount of compute used to train deep learning models has increased 300,000x in six years. Figure taken from Amodei et al.2 This trend is driven by the strong focus of the AI community on obtaining "state-of-the-art" results,b as exemplified by the popularity of leaderboards,53,54 which typically report accuracy (or other similar measures) but omit any mention of cost or efficiency (see, for example, leaderboards.allenai.org).c Despite the clear benefits of improving model accuracy, the focus on this single metric ignores the economic, environmental, and social cost of reaching the reported results.


Will The Next AI Be Superintelligent?

#artificialintelligence

In 2005, Ray Kurzweil said, "the singularity is near." Now, AI can code in any language, and we're moving to way better AI. GPT-3 got "mindboggling" results by training on a ton of data: Basically the whole Internet. It doesn't need to train on your specific use-case (zero-shot learning). It can fool 88% of people, and we're still in the baby stage.


Deep Learning's Climate Change Problem

#artificialintelligence

The human brain is an incredibly efficient source of intelligence. Earlier this month, OpenAI announced it had built the biggest AI model in history. This astonishingly large model, known as GPT-3, is an impressive technical achievement. Yet it highlights a troubling and harmful trend in the field of artificial intelligence--one that has not gotten enough mainstream attention. Modern AI models consume a massive amount of energy, and these energy requirements are growing at a breathtaking rate.


Deep Learning's Climate Change Problem

#artificialintelligence

The human brain is an incredibly efficient source of intelligence. Earlier this month, OpenAI announced it had built the biggest AI model in history. This astonishingly large model, known as GPT-3, is an impressive technical achievement. Yet it highlights a troubling and harmful trend in the field of artificial intelligence--one that has not gotten enough mainstream attention. Modern AI models consume a massive amount of energy, and these energy requirements are growing at a breathtaking rate.