AI performance on demanding benchmarks continues to improve. Performance of advanced AI systems on new benchmarks introduced in 2023 has increased sharply. AI systems also made major strides in generating high-quality video. AI is increasingly embedded in everyday life. In 2023, the FDA (in the US) approved 223 AI-enabled medical devices, up from just six in 2015.

The AI Index Report aims to track, collate, distil, and visualise data related to artificial intelligence. Its mission is "to provide unbiased, rigorously vetted, broadly sourced data in order for policymakers, researchers, executives, journalists, and the general public to develop a more thorough and nuanced understanding of the complex field of AI". The AI Index Report is published on a yearly basis, and the nine-chapter, 500-page 2024 edition has been released today. It investigates trends in research and development, technical performance, responsible AI, and policy and governance. Also considered are the implications of AI systems on the economy, and its use in science and medicine.

More than one-third of researchers believe artificial intelligence (AI) could lead to a "nuclear-level catastrophe", according to a Stanford University survey, underscoring concerns in the sector about the risks posed by the rapidly advancing technology. The survey is among the findings highlighted in the 2023 AI Index Report, released by the Stanford Institute for Human-Centered Artificial Intelligence, which explores the latest developments, risks and opportunities in the burgeoning field of AI. "These systems demonstrate capabilities in question answering, and the generation of text, image, and code unimagined a decade ago, and they outperform the state of the art on many benchmarks, old and new," the report's authors say. "However, they are prone to hallucination, routinely biased, and can be tricked into serving nefarious aims, highlighting the complicated ethical challenges associated with their deployment." The report, which was released earlier this month, comes amid growing calls for regulation of AI following controversies ranging from a chatbot-linked suicide to deepfake videos of Ukrainian President Volodymyr Zelenskyy appearing to surrender to invading Russian forces. Last month, Elon Musk and Apple co-founder Steve Wozniak were among 1,300 signatories of an open letter calling for a six-month pause on training AI systems beyond the level of Open AI's chatbot GPT-4 as "powerful AI systems should be developed only once we are confident that their effects will be positive and their risks will be manageable".

Industry releases more "significant" machine learning models than academia, with 32 released in 2022 compared with three. Until 2014, most significant machine learning models were released by academic institutions. Performance on traditional benchmarks is saturating. State-of-the-art results continue to be published, but year-over-year improvement on many benchmarks continues to be marginal. Training of AI models has environmental implications, with BLOOM's training run emitting 25 times more carbon than a single air traveller on a one-way trip from New York to San Francisco.

The annual report keeps track, collects e displays AI-related data, to support meaningful decisions, and advance AI responsibly and ethically. The AI Index Report supports many different organizations to track progress in artificial intelligence. These organizations include: the Center for Security and Emerging Technology at Georgetown University, LinkedIn, NetBase Quid, Lightcast, and McKinsey. The AI Index Report also expanded its tracking of global AI legislation from 25 countries in 2022 to 127 in 2023. The demand for AI-related job skills is increasing in virtually all industries (in the US).

The AI Index serves as one of the most credible and authoritative sources for data and insights about AI to provide policymakers, researchers, journalists, executives, and the general public a deeper understanding of the field.

Computer vision has escalated its impact on artificial intelligence research, according to the 2022 AI Index Report from Stanford University. In fact, the report calls out an increased interest in "computer vision subtasks, such as medical image segmentation and masked-face identification." But this shift in focus may mean a movement toward more practical applications, something the industry will see more of this year. As AI's intersection with computer science and engineering disciplines continues to climb, so, too, do the complications surrounding its implementation and use. At CVPR 2022, Josh Tennebaum, professor in the Department of Brain and Cognitive Sciences at MIT, discussed the ways the human brain processes information and how that experience extends beyond data inputs and evaluations.

Artificial intelligence is today's most discussed and debated technology, generating widespread adulation and anxiety, and significant government and business interest and investments. But six years after DeepMind's AlphaGo defeated a Go champion, countless research papers showing AI's superior performance over humans in a variety of tasks, and numerous surveys reporting rapid adoption, what is the actual business impact of AI? "2021 was the year that AI went from an emerging technology to a mature technology... that has real-world impact, both positive and negative," declared the 2022 AI Index Report. The 5th installment of the index measures the growing impact of AI in a number of ways, including private investment in AI, the number of AI patents filed, and the number of bills related to AI that were passed into law in legislatures of 25 countries around the world. There is nothing in the report, however, about "real-world impact" as I would define it--measurably successful, long-lasting and significant deployments of AI. There is also no definition of "AI" in the report. Going back to the first installment of the AI Index report, published in 2017, still does not yield a definition of what the report is all about.

In October 2021, Cynthia Rudin was announced as the winner of the AAAI Squirrel AI award, which was formally presented during an award ceremony at AAAI 2022. Following the presentation, Cynthia delivered an invited talk about her research on interpretable machine learning.

The 2022 AI Index Report has been published. Compiled by the Stanford Institute for Human-Centered Artificial Intelligence (HAI), it tracks, summarises and visualises data relating to artificial intelligence. The aim of the report is to provide unbiased, rigorously vetted, and globally sourced data for policymakers, researchers, executives, journalists, and the general public to develop intuitions about the complex field of AI. Find out more about the report here. You can access the full pdf version here.