Collaborating Authors


Neural forecasting at scale Machine Learning

We study the problem of efficiently scaling ensemble-based deep neural networks for time series (TS) forecasting on a large set of time series. Current state-of-the-art deep ensemble models have high memory and computational requirements, hampering their use to forecast millions of TS in practical scenarios. We propose N-BEATS(P), a global multivariate variant of the N-BEATS model designed to allow simultaneous training of multiple univariate TS forecasting models. Our model addresses the practical limitations of related models, reducing the training time by half and memory requirement by a factor of 5, while keeping the same level of accuracy. We have performed multiple experiments detailing the various ways to train our model and have obtained results that demonstrate its capacity to support zero-shot TS forecasting, i.e., to train a neural network on a source TS dataset and deploy it on a different target TS dataset without retraining, which provides an efficient and reliable solution to forecast at scale even in difficult forecasting conditions.

A Plug-and-Play Method for Controlled Text Generation Artificial Intelligence

Large pre-trained language models have repeatedly shown their ability to produce fluent text. Yet even when starting from a prompt, generation can continue in many plausible directions. Current decoding methods with the goal of controlling generation, e.g., to ensure specific words are included, either require additional models or fine-tuning, or work poorly when the task at hand is semantically unconstrained, e.g., story generation. In this work, we present a plug-and-play decoding method for controlled language generation that is so simple and intuitive, it can be described in a single sentence: given a topic or keyword, we add a shift to the probability distribution over our vocabulary towards semantically similar words. We show how annealing this distribution can be used to impose hard constraints on language generation, something no other plug-and-play method is currently able to do with SOTA language generators. Despite the simplicity of this approach, we see it works incredibly well in practice: decoding from GPT-2 leads to diverse and fluent sentences while guaranteeing the appearance of given guide words. We perform two user studies, revealing that (1) our method outperforms competing methods in human evaluations; and (2) forcing the guide words to appear in the generated text has no impact on the fluency of the generated text.

Graph Learning for Cognitive Digital Twins in Manufacturing Systems Artificial Intelligence

Future manufacturing requires complex systems that connect simulation platforms and virtualization with physical data from industrial processes. Digital twins incorporate a physical twin, a digital twin, and the connection between the two. Benefits of using digital twins, especially in manufacturing, are abundant as they can increase efficiency across an entire manufacturing life-cycle. The digital twin concept has become increasingly sophisticated and capable over time, enabled by rises in many technologies. In this paper, we detail the cognitive digital twin as the next stage of advancement of a digital twin that will help realize the vision of Industry 4.0. Cognitive digital twins will allow enterprises to creatively, effectively, and efficiently exploit implicit knowledge drawn from the experience of existing manufacturing systems. They also enable more autonomous decisions and control, while improving the performance across the enterprise (at scale). This paper presents graph learning as one potential pathway towards enabling cognitive functionalities in manufacturing digital twins. A novel approach to realize cognitive digital twins in the product design stage of manufacturing that utilizes graph learning is presented.

Survey XII: What Is the Future of Ethical AI Design? – Imagining the Internet


Results released June 16, 2021 – Pew Research Center and Elon University's Imagining the Internet Center asked experts where they thought efforts aimed at ethical artificial intelligence design would stand in the year 2030. Some 602 technology innovators, developers, business and policy leaders, researchers and activists responded to this specific question. The Question – Regarding the application of AI Ethics by 2030: In recent years, there have been scores of convenings and even more papers generated proposing ethical frameworks for the application of artificial intelligence (AI). They cover a host of issues including transparency, justice and fairness, privacy, freedom and human autonomy, beneficence and non-maleficence, freedom, trust, sustainability and dignity. Our questions here seek your predictions about the possibilities for such efforts. By 2030, will most of the AI systems being used by organizations of all sorts employ ethical principles focused primarily on the public ...

Towards Explainable Fact Checking Machine Learning

The past decade has seen a substantial rise in the amount of mis- and disinformation online, from targeted disinformation campaigns to influence politics, to the unintentional spreading of misinformation about public health. This development has spurred research in the area of automatic fact checking, from approaches to detect check-worthy claims and determining the stance of tweets towards claims, to methods to determine the veracity of claims given evidence documents. These automatic methods are often content-based, using natural language processing methods, which in turn utilise deep neural networks to learn higher-order features from text in order to make predictions. As deep neural networks are black-box models, their inner workings cannot be easily explained. At the same time, it is desirable to explain how they arrive at certain decisions, especially if they are to be used for decision making. While this has been known for some time, the issues this raises have been exacerbated by models increasing in size, and by EU legislation requiring models to be used for decision making to provide explanations, and, very recently, by legislation requiring online platforms operating in the EU to provide transparent reporting on their services. Despite this, current solutions for explainability are still lacking in the area of fact checking. This thesis presents my research on automatic fact checking, including claim check-worthiness detection, stance detection and veracity prediction. Its contributions go beyond fact checking, with the thesis proposing more general machine learning solutions for natural language processing in the area of learning with limited labelled data. Finally, the thesis presents some first solutions for explainable fact checking.

On the Opportunities and Risks of Foundation Models Artificial Intelligence

AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks. We call these models foundation models to underscore their critically central yet incomplete character. This report provides a thorough account of the opportunities and risks of foundation models, ranging from their capabilities (e.g., language, vision, robotics, reasoning, human interaction) and technical principles(e.g., model architectures, training procedures, data, systems, security, evaluation, theory) to their applications (e.g., law, healthcare, education) and societal impact (e.g., inequity, misuse, economic and environmental impact, legal and ethical considerations). Though foundation models are based on standard deep learning and transfer learning, their scale results in new emergent capabilities,and their effectiveness across so many tasks incentivizes homogenization. Homogenization provides powerful leverage but demands caution, as the defects of the foundation model are inherited by all the adapted models downstream. Despite the impending widespread deployment of foundation models, we currently lack a clear understanding of how they work, when they fail, and what they are even capable of due to their emergent properties. To tackle these questions, we believe much of the critical research on foundation models will require deep interdisciplinary collaboration commensurate with their fundamentally sociotechnical nature.

The Role of Social Movements, Coalitions, and Workers in Resisting Harmful Artificial Intelligence and Contributing to the Development of Responsible AI Artificial Intelligence

There is mounting public concern over the influence that AI based systems has in our society. Coalitions in all sectors are acting worldwide to resist hamful applications of AI. From indigenous people addressing the lack of reliable data, to smart city stakeholders, to students protesting the academic relationships with sex trafficker and MIT donor Jeffery Epstein, the questionable ethics and values of those heavily investing in and profiting from AI are under global scrutiny. There are biased, wrongful, and disturbing assumptions embedded in AI algorithms that could get locked in without intervention. Our best human judgment is needed to contain AI's harmful impact. Perhaps one of the greatest contributions of AI will be to make us ultimately understand how important human wisdom truly is in life on earth.

Blavatnik Family Foundation, New York Academy of Sciences Name 31 Finalists for 2021 Blavatnik National Awards for Young Scientists

CMU School of Computer Science

Showcasing America's most promising young scientists and engineers, the Blavatnik Family Foundation and the New York Academy of Sciences today named 31 finalists for the world's largest unrestricted prize honoring early-career scientists and engineers. Three winners of the Blavatnik National Awards for Young Scientists – in life sciences, chemistry, and physical sciences and engineering – will be announced on July 20, each receiving $250,000 as a Blavatnik National Awards Laureate. The finalists, culled from 298 nominations by 157 United States research institutions across 38 states, have made trailblazing discoveries in wide-ranging fields, from the neuroscience of addiction to the development of gene-editing technologies, from designing next-generation battery storage to understanding the origins of photosynthesis, from making improvements in computer vision to pioneering new frontiers in polymer chemistry. Descriptions of the honorees' research are listed below. "Each day, young scientists tirelessly seek solutions to humanity's greatest challenges," said Len Blavatnik, founder and chairman of Access Industries, and head of the Blavatnik Family Foundation. "The Blavatnik Awards recognize this scientific brilliance and tenacity as we honor these 31 finalists. We congratulate them on their accomplishments and look forward to their continued, future discoveries and success." President and CEO of the New York Academy of Sciences Nicholas B. Dirks said: "Each year, it is a complete joy to see the very'best of the best' of American science represented by the Blavatnik National Awards Finalists." spotlights 21 projects focused on healthcare, energy efficiency and climate security

ZDNet's Digital Transformation Institute announced the 21 winners of their contest centered around healthcare, energy and climate-related projects. The company offered between $100,000 and $250,000 to groups that could start projects using AI and digital transformation to address COVID-19, climate security and energy efficiency. Out of the 52 submissions that came in since February, 21 were selected for the grants, with each focusing on efforts to "improve resilience, sustainability, and efficiency" using "carbon sequestration, carbon markets, hydrocarbon production, distributed renewables, and cybersecurity." S. Shankar Sastry, a co-director of DTI and a leading computer science professor at the University of California, Berkeley, said the world was now being threatened by the pandemic, powerful wildfires, rising seas, monster storms and other severe weather threats. Marta Gonzalez, an associate professor at the University of California, Berkeley, is looking to create a platform that could collate more data about wildfires. The project will involve "crowdsourcing and very high-resolution remote sensing for an AI-driven fuel model identification; models of wildfire behavior, intensity, spread, informed by downscaled climate change predictions, historic catastrophic wildfires, environmental monitoring; and egress models that combine large-scale mobile phone data facilitated by data-driven optimization models and computation."

Exploring the future of humanitarian technology


Deb Campbell, a senior staff member in the HADR Systems Group, started the session with a discussion of how to accelerate the national and global response to climate change. "Because the timeline is so short and challenges so complex, it is essential to make good, evidence-based decisions on how to get to where we need to go," she said. "We call this approach systems analysis and architecture, and by taking this approach we can create a national climate change resilience roadmap." This roadmap implements more of what we already know how to do, for example utilizing wind and solar energy, and identifies gaps where research and development are needed to reach specific goals. One example is the transition to a fully zero-emission vehicle (ZEV) fleet in the United States in the coming decades; California has already directed that all of the state's new car sales be ZEV by 2035.