Goto

Collaborating Authors

 Kainuu


AERIS: Argonne Earth Systems Model for Reliable and Skillful Predictions

Hatanpää, Väinö, Ku, Eugene, Stock, Jason, Emani, Murali, Foreman, Sam, Jung, Chunyong, Madireddy, Sandeep, Nguyen, Tung, Sastry, Varuni, Sinurat, Ray A. O., Wheeler, Sam, Zheng, Huihuo, Arcomano, Troy, Vishwanath, Venkatram, Kotamarthi, Rao

arXiv.org Artificial Intelligence

Generative machine learning offers new opportunities to better understand complex Earth system dynamics. Recent diffusion-based methods address spectral biases and improve ensemble calibration in weather forecasting compared to deterministic methods, yet have so far proven difficult to scale stably at high resolutions. We introduce AERIS, a 1.3 to 80B parameter pixel-level Swin diffusion transformer to address this gap, and SWiPe, a generalizable technique that composes window parallelism with sequence and pipeline parallelism to shard window-based transformers without added communication cost or increased global batch size. On Aurora (10,080 nodes), AERIS sustains 10.21 ExaFLOPS (mixed precision) and a peak performance of 11.21 ExaFLOPS with $1 \times 1$ patch size on the 0.25° ERA5 dataset, achieving 95.5% weak scaling efficiency, and 81.6% strong scaling efficiency. AERIS outperforms the IFS ENS and remains stable on seasonal scales to 90 days, highlighting the potential of billion-parameter diffusion models for weather and climate prediction.


Open-source Swiss language model to be released this summer

AIHub

This summer, EPFL and ETH Zurich will release a large language model (LLM) developed on public infrastructure. Trained on the "Alps" supercomputer at the Swiss National Supercomputing Centre (CSCS), the new LLM marks a milestone in open-source AI and multilingual excellence. Earlier this month in Geneva, around 50 leading global initiatives and organisations dedicated to open-source LLMs and trustworthy AI convened at the International Open-Source LLM Builders Summit. Hosted by the AI centres of EPFL and ETH Zurich, the event marked a significant step in building a vibrant and collaborative international ecosystem for open foundation models. Open LLMs are increasingly viewed as credible alternatives to commercial systems, most of which are developed behind closed doors in the United States or China.


Aurora-M: The First Open Source Multilingual Language Model Red-teamed according to the U.S. Executive Order

Nakamura, Taishi, Mishra, Mayank, Tedeschi, Simone, Chai, Yekun, Stillerman, Jason T, Friedrich, Felix, Yadav, Prateek, Laud, Tanmay, Chien, Vu Minh, Zhuo, Terry Yue, Misra, Diganta, Bogin, Ben, Vu, Xuan-Son, Karpinska, Marzena, Dantuluri, Arnav Varma, Kusa, Wojciech, Furlanello, Tommaso, Yokota, Rio, Muennighoff, Niklas, Pai, Suhas, Adewumi, Tosin, Laippala, Veronika, Yao, Xiaozhe, Junior, Adalberto, Ariyak, Alpay, Drozd, Aleksandr, Clive, Jordan, Gupta, Kshitij, Chen, Liangyu, Sun, Qi, Tsui, Ken, Persaud, Noah, Fahmy, Nour, Chen, Tianlong, Bansal, Mohit, Monti, Nicolo, Dang, Tai, Luo, Ziyang, Bui, Tien-Tung, Navigli, Roberto, Mehta, Virendra, Blumberg, Matthew, May, Victor, Nguyen, Huu, Pyysalo, Sampo

arXiv.org Artificial Intelligence

Pretrained language models underpin several AI applications, but their high computational cost for training limits accessibility. Initiatives such as BLOOM and StarCoder aim to democratize access to pretrained models for collaborative community development. However, such existing models face challenges: limited multilingual capabilities, continual pretraining causing catastrophic forgetting, whereas pretraining from scratch is computationally expensive, and compliance with AI safety and development laws. This paper presents Aurora-M, a 15B parameter multilingual open-source model trained on English, Finnish, Hindi, Japanese, Vietnamese, and code. Continually pretrained from StarCoderPlus on 435 billion additional tokens, Aurora-M surpasses 2 trillion tokens in total training token count. It is the first open-source multilingual model fine-tuned on human-reviewed safety instructions, thus aligning its development not only with conventional red-teaming considerations, but also with the specific concerns articulated in the Biden-Harris Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence. Aurora-M is rigorously evaluated across various tasks and languages, demonstrating robustness against catastrophic forgetting and outperforming alternatives in multilingual settings, particularly in safety evaluations. To promote responsible open-source LLM development, Aurora-M and its variants are released at https://huggingface.co/collections/aurora-m/aurora-m-models-65fdfdff62471e09812f5407 .


Eva-KELLM: A New Benchmark for Evaluating Knowledge Editing of LLMs

Wu, Suhang, Peng, Minlong, Chen, Yue, Su, Jinsong, Sun, Mingming

arXiv.org Artificial Intelligence

Large language models (LLMs) possess a wealth of knowledge encoded in their parameters. However, this knowledge may become outdated or unsuitable over time. As a result, there has been a growing interest in knowledge editing for LLMs and evaluating its effectiveness. Existing studies primarily focus on knowledge editing using factual triplets, which not only incur high costs for collection but also struggle to express complex facts. Furthermore, these studies are often limited in their evaluation perspectives. In this paper, we propose Eva-KELLM, a new benchmark for evaluating knowledge editing of LLMs. This benchmark includes an evaluation framework and a corresponding dataset. Under our framework, we first ask the LLM to perform knowledge editing using raw documents, which provides a more convenient and universal approach compared to using factual triplets. We then evaluate the updated LLM from multiple perspectives. In addition to assessing the effectiveness of knowledge editing and the retention of unrelated knowledge from conventional studies, we further test the LLM's ability in two aspects: 1) Reasoning with the altered knowledge, aiming for the LLM to genuinely learn the altered knowledge instead of simply memorizing it. 2) Cross-lingual knowledge transfer, where the LLM updated with raw documents in one language should be capable of handling queries from another language. To facilitate further research, we construct and release the corresponding dataset. Using this benchmark, we investigate the effectiveness of several commonly-used knowledge editing methods. Experimental results indicate that the current methods for knowledge editing using raw documents are not effective in yielding satisfactory results, particularly when it comes to reasoning with altered knowledge and cross-lingual knowledge transfer.


Europe's fastest supercomputer is now connected to a quantum computer

New Scientist

A quantum computer has been connected to Europe's fastest supercomputer. It may be a step towards a new type of computing that combines traditional and quantum computers to quickly solve complex problems. The promise of quantum computers is that they will eventually complete calculations that are impossible for the most powerful conventional computers. Though many researchers are working on perfecting quantum computers, many are also suggesting that existing, imperfect quantum computers could be more useful if connected to traditional supercomputers.


US's Frontier supercomputer becomes the fastest in the world

Daily Mail - Science & tech

A supercomputer in the US called'Frontier' has become the fastest in the world, beating its closest rival in Japan. Frontier, based at the US Department of Energy's Oak Ridge National Laboratory in Tennessee, is the first to achieve a level of computing known as'exascale'. Exascale refers to a system that can perform at least one quintillion operations per second – a billion billion calculations, or 1 followed by 18 zeroes. This makes Frontier more than twice as powerful than the Fugaku supercomputer in Japan, which was deemed the world's fastest supercomputer back in June 2020. Frontier will allow scientists to develop technologies for the US's energy, economic and national security, said Oak Ridge National Laboratory, and solve computational problems that were impossible to do just five years ago.


A Comic Walks Into a VR Comedy Club…

WSJ.com: WSJD - Technology

Samantha Gilweit, who is used to performing improv comedy in front of large crowds, went with her best opening bit, the one about drunken princesses that never fails. She delivered the punchline, and…dead silence. Her mind raced with how to recover. A second later, smiley-face emoji appeared over the heads of the animated robots and digitally rendered humanoids that stood in for the audience, followed by a gush of hearts. That's how "you knew you were killing it," the 31-year-old said.