Get the Slate Quiz in your inbox every weekday. You can manage your newsletter subscriptions at any time. You're already subscribed to the Slate Games newsletter. You can manage your newsletter subscriptions at any time. We encountered an issue signing you up.

Low-Rank Adaptation (LoRA) fine-tunes large models by learning low-rank updates on top of frozen weights, dramatically reducing trainable parameters and memory. However, there is still a gap between full training with low-rank projections (SVDLoRA) and LoRA fine-tuning, indicating that LoRA steps can be further improved. In this study, we propose OPLoRA, a memory-efficient optimizer that closes this gap by casting LoRA optimization as an interpretable sub-problem and solving it efficiently with alternating least squares updates, where 1-2 alternating steps are empirically found to be sufficient to closely match truncated SVD without ever forming the full matrix. We also retrieve the recently proposed preconditioning methods for LoRA as a special case. OPLoRA supports momentum by maintaining a low-rank estimate using the same subroutine (LoRSum) for computing the step, with a memory budget of 3 times the number of LoRA parameters (i.e., same as Adam). We also propose an experimental scaled variant that uses the K-FAC metric, which could be of interest. Across a linear task, MNIST, CIFAR-100, and RoBERTa-base (MNLI), OPLoRA consistently approaches SVDLoRA's performance using significantly less memory.

Get the Slate Quiz in your inbox every weekday. You can manage your newsletter subscriptions at any time. You're already subscribed to the Slate Games newsletter. You can manage your newsletter subscriptions at any time. We encountered an issue signing you up.

Welcome to Slate's weekly news quiz. It's Friday, which means it's time to test your knowledge of the week's news events. Your host, Ray Hamel, has concocted questions on news topics ranging from politics to business, from culture to sports to science. At the end of the quiz, you'll be able to compare your score with that of the average contestant, as well as that of a Slatester who has agreed to take the quiz on the record. This week's contestant is senior editor Rebecca Onion.

--The rapid adoption of generative AI (GenAI), particularly Large Language Models (LLMs), has exposed critical limitations of cloud-centric deployments, including latency, cost, and privacy concerns. Meanwhile, Small Language Models (SLMs) are emerging as viable alternatives for resource-constrained edge environments, though they often lack the capabilities of their larger counterparts. This article explores the potential of collaborative inference systems that leverage both edge and cloud resources to address these challenges. By presenting distinct cooperation strategies alongside practical design principles and experimental insights, we offer actionable guidance for deploying GenAI across the computing continuum. Ultimately, this work underscores the great potential of edge-first approaches in realizing the promise of GenAI in diverse, real-world applications. It is no longer necessary to elaborate extensively on the transformative impact of generative AI (GenAI) models, particularly Large Language Models (LLMs), across various sectors of society. From healthcare to education, entertainment to software development and IoT [1], it is evident that nearly every application domain is ready (or already is) to be influenced by these technologies. LLMs like GPT -4, powered by transformer architectures with billions of parameters, excel in diverse NLP tasks (e.g., summarization, translation, query answering) and high-level reasoning.

Test-time scaling has significantly improved large language model performance, enabling deeper reasoning to solve complex problems. However, this increased reasoning capability also leads to excessive token generation and unnecessary problem-solving attempts. We introduce Don\'t Answer Bench (DNA Bench), a new benchmark designed to evaluate LLMs ability to robustly understand the tricky reasoning triggers and avoiding unnecessary generation. DNA Bench consists of 150 adversarially designed prompts that are easy for humans to understand and respond to, but surprisingly not for many of the recent prominent LLMs. DNA Bench tests models abilities across different capabilities, such as instruction adherence, hallucination avoidance, redundancy filtering, and unanswerable question recognition. We evaluate reasoning LLMs (RLMs), including DeepSeek-R1, OpenAI O3-mini, Claude-3.7-sonnet and compare them against a powerful non-reasoning model, e.g., GPT-4o. Our experiments reveal that RLMs generate up to 70x more tokens than necessary, often failing at tasks that simpler non-reasoning models handle efficiently with higher accuracy. Our findings underscore the need for more effective training and inference strategies in RLMs.

You can manage your newsletter subscriptions at any time. As is now tradition, the final quiz of the year is a look back at the past 12 months. It was a year fraught with discord, so grab your favorite beverage, maybe a cookie or two, and take a deep, relaxing breath before you plunge into 2023 for one last time in this week's Slate News Quiz. If this is your first time playing, read the rules here. The quiz may require you to turn on cookies in your browser for it to function properly.

The introduction of large language models (LLMs) like ChatGPT and Google Palm2 for smart contract generation seems to be the first well-established instance of an AI pair programmer. LLMs have access to a large number of open-source smart contracts, enabling them to utilize more extensive code in Solidity than other code generation tools. Although the initial and informal assessments of LLMs for smart contract generation are promising, a systematic evaluation is needed to explore the limits and benefits of these models. The main objective of this study is to assess the quality of generated code provided by LLMs for smart contracts. We also aim to evaluate the impact of the quality and variety of input parameters fed to LLMs. To achieve this aim, we created an experimental setup for evaluating the generated code in terms of validity, correctness, and efficiency. Our study finds crucial evidence of security bugs getting introduced in the generated smart contracts as well as the overall quality and correctness of the code getting impacted. However, we also identified the areas where it can be improved. The paper also proposes several potential research directions to improve the process, quality and safety of generated smart contract codes.

Researchers at USC are developing an algorithm that teaches machines to learn without human supervision. "Generally speaking, machine learning is the science of teaching machines to act like humans," said Mohammad Rostami, Research Lead at USC Viterbi's Information Sciences Institute (ISI). Teaching machines to learn without human supervision is the subject of his latest paper, Overcoming Concept Shift in Domain-Aware Settings through Consolidated Internal Distributions, which he will present at the 37th AAAI Conference on Artificial Intelligence, held in Washington, D.C. Rostami explained how machine learning is typically done: "We collect data annotated by humans, and then we teach the machine how to act similar to humans given that data. The problem is that the knowledge the machine obtains is limited to the data set used for training." Additionally, the training data set is often unavailable after the training process is complete.

The use of cloud technology has become a mainstream practice in today's digital age. It has enabled businesses and individuals to store, share, and access data and applications from anywhere in the world. However, managing the cloud has become complex and challenging with the increasing volume of daily data. This is where artificial intelligence (AI) comes in. AI is revolutionizing how businesses manage their cloud infrastructure by enabling them to automate processes, identify trends, and make more informed decisions.