Large Language Model
How Do Companies Manage the Environmental Sustainability of AI? An Interview Study About Green AI Efforts and Regulations
Sampatsing, Ashmita, Vos, Sophie, Beauxis-Aussalet, Emma, Bogner, Justus
With the ever-growing adoption of artificial intelligence (AI), AI-based software and its negative impact on the environment are no longer negligible, and studying and mitigating this impact has become a critical area of research. However, it is currently unclear which role environmental sustainability plays during AI adoption in industry and how AI regulations influence Green AI practices and decision-making in industry. We therefore aim to investigate the Green AI perception and management of industry practitioners. To this end, we conducted a total of 11 interviews with participants from 10 different organizations that adopted AI-based software. The interviews explored three main themes: AI adoption, current efforts in mitigating the negative environmental impact of AI, and the influence of the EU AI Act and the Corporate Sustainability Reporting Directive (CSRD). Our findings indicate that 9 of 11 participants prioritized business efficiency during AI adoption, with minimal consideration of environmental sustainability. Monitoring and mitigation of AI's environmental impact were very limited. Only one participant monitored negative environmental effects. Regarding applied mitigation practices, six participants reported no actions, with the others sporadically mentioning techniques like prompt engineering, relying on smaller models, or not overusing AI. Awareness and compliance with the EU AI Act are low, with only one participant reporting on its influence, while the CSRD drove sustainability reporting efforts primarily in larger companies. All in all, our findings reflect a lack of urgency and priority for sustainable AI among these companies. We suggest that current regulations are not very effective, which has implications for policymakers. Additionally, there is a need to raise industry awareness, but also to provide user-friendly techniques and tools for Green AI practices.
A Survey on Inference Engines for Large Language Models: Perspectives on Optimization and Efficiency
Park, Sihyeong, Jeon, Sungryeol, Lee, Chaelyn, Jeon, Seokhun, Kim, Byung-Soo, Lee, Jemin
Large language models (LLMs) are widely applied in chatbots, code generators, and search engines. Workload such as chain-of-throught, complex reasoning, agent services significantly increase the inference cost by invoke the model repeatedly. Optimization methods such as parallelism, compression, and caching have been adopted to reduce costs, but the diverse service requirements make it hard to select the right method. Recently, specialized LLM inference engines have emerged as a key component for integrating the optimization methods into service-oriented infrastructures. However, a systematic study on inference engines is still lacking.This paper provides a comprehensive evaluation of 25 open-source and commercial inference engines. We examine each inference engine in terms of ease-of-use, ease-of-deployment, general-purpose support, scalability, and suitability for throughput- and latency-aware computation. Furthermore, we explore the design goals of each inference engine by investigating the optimization techniques it supports. In addition, we assess the ecosystem maturity of open source inference engines and handle the performance and cost policy of commercial solutions.We outline future research directions that include support for complex LLM-based services, support of various hardware, and enhanced security, offering practical guidance to researchers and developers in selecting and designing optimized LLM inference engines. We also provide a public repository to continually track developments in this fast-evolving field: \href{https://github.com/sihyeong/Awesome-LLM-Inference-Engine}{https://github.com/sihyeong/Awesome-LLM-Inference-Engine}.
MigGPT: Harnessing Large Language Models for Automated Migration of Out-of-Tree Linux Kernel Patches Across Versions
Dang, Pucheng, Huang, Di, Li, Dong, Chen, Kang, Wen, Yuanbo, Guo, Qi, Hu, Xing
Out-of-tree kernel patches are essential for adapting the Linux kernel to new hardware or enabling specific functionalities. Maintaining and updating these patches across different kernel versions demands significant effort from experienced engineers. Large language models (LLMs) have shown remarkable progress across various domains, suggesting their potential for automating out-of-tree kernel patch migration. However, our findings reveal that LLMs, while promising, struggle with incomplete code context understanding and inaccurate migration point identification. In this work, we propose MigGPT, a framework that employs a novel code fingerprint structure to retain code snippet information and incorporates three meticulously designed modules to improve the migration accuracy and efficiency of out-of-tree kernel patches. Furthermore, we establish a robust benchmark using real-world out-of-tree kernel patch projects to evaluate LLM capabilities. Evaluations show that MigGPT significantly outperforms the direct application of vanilla LLMs, achieving an average completion rate of 74.07 for migration tasks.
Reasoning Transfer for an Extremely Low-Resource and Endangered Language: Bridging Languages Through Sample-Efficient Language Understanding
Tran, Khanh-Tung, O'Sullivan, Barry, Nguyen, Hoang D.
Recent advances have enabled Large Language Models (LLMs) to tackle reasoning tasks by generating chain-of-thought (CoT) rationales, yet these gains have largely applied to high-resource languages, leaving low-resource languages behind. In this work, we first investigate CoT techniques in extremely low-resource scenarios through previous prompting, model-editing, and fine-tuning approaches. We introduce English-Pivoted CoT Training, leveraging the insight that LLMs internally operate in a latent space aligned toward the dominant language. Given input in a low-resource language, we perform supervised fine-tuning to generate CoT in English and output the final response in the target language. Across mathematical reasoning benchmarks, our approach outperforms other baselines with up to 28.33% improvement in low-resource scenarios. Our analysis and additional experiments, including Mixed-Language CoT and Two-Stage Training, show that explicitly separating language understanding from reasoning enhances cross-lingual reasoning abilities. To facilitate future work, we also release \emph{LC2024}, the first benchmark for mathematical tasks in Irish, an extremely low-resource and endangered language. Our results and resources highlight a practical pathway to multilingual reasoning without extensive retraining in every extremely low-resource language, despite data scarcity.
Jeff Bezos' New AI Venture Quietly Acquired an Agentic Computing Startup
Jeff Bezos' New AI Venture Quietly Acquired an Agentic Computing Startup Project Prometheus has raised over $6 billion in funding and hired over 100 employees, a handful of whom joined through its acquisition of General Agents, according to records and sources. In early June, tech entrepreneur Vik Bajaj took over Saison, a two-Michelin star restaurant in San Francisco, for an off-the-record dinner to talk about AI with journalists and a handful of scientists. In attendance was Sherjil Ozair, a late addition who had previously held senior research roles at DeepMind and Tesla . The following day, Bajaj and Ozair were on their way to making a deal, public records show. Bajaj didn't mention it at the dinner, but earlier this year he had begun working with Amazon executive chairman Jeff Bezos on a new AI venture called Project Prometheus.
ChatGPT firm blames boy's suicide on 'misuse' of its technology
Adam Raine's family say the version of ChatGPT he used had'clear safety issues'. Adam Raine's family say the version of ChatGPT he used had'clear safety issues'. ChatGPT firm blames boy's suicide on'misuse' of its technology The maker of ChatGPT has said the suicide of a 16-year-old was down to his "misuse" of its system and was "not caused" by the chatbot. The comments came in OpenAI's response to a lawsuit filed against the San Francisco company and its chief executive, Sam Altman, by the family of California teenager Adam Raine. Raine killed himself in April after extensive conversations and "months of encouragement from ChatGPT", the family's lawyer has said.
The Download: AI and the economy, and slop for the masses
There's a lot at stake when it comes to understanding how AI is changing the economy right now. Or is the situation too nuanced for that? Hopefully, we can point you towards some answers. Mat Honan, our editor in chief, will hold a special subscriber-only Roundtables conversation with our editor at large David Rotman, and Richard Waters, columnist, exploring what's happening across different markets. Register here to join us at 1pm ET on Tuesday December 9. The event is part of the and "The State of AI" partnership, exploring the global impact of artificial intelligence.
Welcome to the Slopverse
Listen to more stories on the Noa app. Bill Lowery, a sales executive, is confused when a workmate asks where he should take a date out for dinosaur. "You're planning to take this girl out for?" "That's right," the colleague responds, totally nonchalant. Lowery presses him, agitated: "Wait a minute. What is this, some sort of new-wave expression or something--saying instead of?" "He's so pale and awfully congested--and he didn't touch his dinosaur when I took it in to him."
The AI Hype Index: The people can't get enough of AI slop
The AI Hype Index: The people can't get enough of AI slop That's why we've created the AI Hype Index--a simple, at-a-glance summary of everything you need to know about the state of the industry. Last year, the fantasy author Joanna Maciejewska went viral (if such a thing is still possible on X) with a post saying "I want AI to do my laundry and dishes so that I can do art and writing, not for AI to do my art and writing so that I can do my laundry and dishes." Clearly, it struck a chord with the disaffected masses. Regrettably, 18 months after Maciejewska's post, the entertainment industry insists that machines should make art and artists should do laundry. The streaming platform Disney+ has plans to let its users generate their own content from its intellectual property instead of, y'know, paying humans to make some new Star Wars or Marvel movies. Elsewhere, it seems AI-generated music is resonating with a depressingly large audience, given that the AI band Breaking Rust has topped's Country Digital Song Sales chart.
SoftBank's 40% slide from peak shows worry over giant OpenAI bet
SoftBank shares have plunged around 40% since late October as it sits at the forefront of a global AI selloff. Growing unease over frothy artificial intelligence valuations is weighing on shares of SoftBank Group, which traders increasingly view as a proxy for privately held OpenAI. The Japanese tech investor sits at the forefront of a global AI selloff amid worries about new pressure on OpenAI following Alphabet's Gemini 3.0 debut. SoftBank shares have plunged around 40% since late October, erasing over ¥16 trillion ($102 billion) in market value, as its founder Masayoshi Son prepares to double down on OpenAI and the infrastructure that supports it. SoftBank has ridden the global AI investment boom faster than any other Japanese company.