Goto

Collaborating Authors

 renovation


MemLoRA: Distilling Expert Adapters for On-Device Memory Systems

Bini, Massimo, Bohdal, Ondrej, Michieli, Umberto, Akata, Zeynep, Ozay, Mete, Ceritli, Taha

arXiv.org Artificial Intelligence

Memory-augmented Large Language Models (LLMs) have demonstrated remarkable consistency during prolonged dialogues by storing relevant memories and incorporating them as context. Such memory-based personalization is also key in on-device settings that allow users to keep their conversations and data private. However, memory-augmented systems typically rely on LLMs that are too costly for local on-device deployment. Even though Small Language Models (SLMs) are more suitable for on-device inference than LLMs, they cannot achieve sufficient performance. Additionally, these LLM-based systems lack native visual capabilities, limiting their applicability in multimodal contexts. In this paper, we introduce (i) MemLoRA, a novel memory system that enables local deployment by equipping SLMs with specialized memory adapters, and (ii) its vision extension MemLoRA-V, which integrates small Vision-Language Models (SVLMs) to memory systems, enabling native visual understanding. Following knowledge distillation principles, each adapter is trained separately for specific memory operations$\unicode{x2013}$knowledge extraction, memory update, and memory-augmented generation. Equipped with memory adapters, small models enable accurate on-device memory operations without cloud dependency. On text-only operations, MemLoRA outperforms 10$\times$ larger baseline models (e.g., Gemma2-27B) and achieves performance comparable to 60$\times$ larger models (e.g., GPT-OSS-120B) on the LoCoMo benchmark. To evaluate visual understanding operations instead, we extend LoCoMo with challenging Visual Question Answering tasks that require direct visual reasoning. On this, our VLM-integrated MemLoRA-V shows massive improvements over caption-based approaches (81.3 vs. 23.7 accuracy) while keeping strong performance in text-based tasks, demonstrating the efficacy of our method in multimodal contexts.


When Thinking LLMs Lie: Unveiling the Strategic Deception in Representations of Reasoning Models

Wang, Kai, Zhang, Yihao, Sun, Meng

arXiv.org Artificial Intelligence

The honesty of large language models (LLMs) is a critical alignment challenge, especially as advanced systems with chain-of-thought (CoT) reasoning may strategically deceive humans. Unlike traditional honesty issues on LLMs, which could be possibly explained as some kind of hallucination, those models' explicit thought paths enable us to study strategic deception--goal-driven, intentional misinformation where reasoning contradicts outputs. Using representation engineering, we systematically induce, detect, and control such deception in CoT-enabled LLMs, extracting "deception vectors" via Linear Artificial Tomography (LAT) for 89% detection accuracy. Through activation steering, we achieve a 40% success rate in eliciting context-appropriate deception without explicit prompts, unveiling the specific honesty-related issue of reasoning models and providing tools for trustworthy AI alignment.


In key Congressional race, Republicans criticize Democrat's Central Valley real estate deal

Los Angeles Times

When the federal government closed Castle Air Force Base in Merced County in the 1990s, the dilapidated buildings and vast expanse of aging tarmac left behind seemed more like a liability than an opportunity. But by 2018, the old runways that once carried B-52 bombers had found a new and unexpected customer: Google, which was testing its experimental self-driving vehicles there, far from the prying eyes of Silicon Valley. At the urging of then-state Assemblyman Adam Gray, California gave Merced County 6.5 million that year to expand the self-driving testing program at the old base. A few years later, Gray invested there, too. In 2022, a company in which Gray is a minority owner bought four apartment buildings on the former base from Merced County, according to a Times review of business filings, property records and Gray's financial disclosures.


Enhancing the Efficiency and Accuracy of Underlying Asset Reviews in Structured Finance: The Application of Multi-agent Framework

Wan, Xiangpeng, Deng, Haicheng, Zou, Kai, Xu, Shiqi

arXiv.org Artificial Intelligence

Structured finance, which involves restructuring diverse assets into securities like MBS, ABS, and CDOs, enhances capital market efficiency but presents significant due diligence challenges. This study explores the integration of artificial intelligence (AI) with traditional asset review processes to improve efficiency and accuracy in structured finance. Using both open-sourced and close-sourced large language models (LLMs), we demonstrate that AI can automate the verification of information between loan applications and bank statements effectively. While close-sourced models such as GPT-4 show superior performance, open-sourced models like LLAMA3 offer a cost-effective alternative. Dual-agent systems further increase accuracy, though this comes with higher operational costs. This research highlights AI's potential to minimize manual errors and streamline due diligence, suggesting a broader application of AI in financial document analysis and risk management.


Novel Preprocessing Technique for Data Embedding in Engineering Code Generation Using Large Language Model

Lin, Yu-Chen, Kumar, Akhilesh, Chang, Norman, Zhang, Wenliang, Zakir, Muhammad, Apte, Rucha, He, Haiyang, Wang, Chao, Jang, Jyh-Shing Roger

arXiv.org Artificial Intelligence

We present four main contributions to enhance the performance of Large Language Models (LLMs) in generating domain-specific code: (i) utilizing LLM-based data splitting and data renovation techniques to improve the semantic representation of embeddings' space; (ii) introducing the Chain of Density for Renovation Credibility (CoDRC), driven by LLMs, and the Adaptive Text Renovation (ATR) algorithm for assessing data renovation reliability; (iii) developing the Implicit Knowledge Expansion and Contemplation (IKEC) Prompt technique; and (iv) effectively refactoring existing scripts to generate new and high-quality scripts with LLMs. By using engineering simulation software RedHawk-SC as a case study, we demonstrate the effectiveness of our data pre-processing method for expanding and categorizing scripts. When combined with IKEC, these techniques enhance the Retrieval-Augmented Generation (RAG) method in retrieving more relevant information, ultimately achieving a 73.33% "Percentage of Correct Lines" for code generation problems in MapReduce applications.


Artificial Intelligence and Industry 4.0

#artificialintelligence

Much of the hype nearby artificial intelligence in manufacturing is focused on industrial automation, on the other hand, this is just one aspect of the smart factory revolution -- a natural next step in the chase of efficiency. What artificial intelligence additionally brings to the producing desk is its functionality to open up absolutely new avenues for business. Below is a precis that covers each of those factors of artificial intelligence in the Industry 4.0 paradigm, and the way this effective generation is already being utilized by producers to pressure performance, and enhance great and higher control delivery chains. Artificial intelligence's effect on production can be prepared into five fundamental areas: Reducing manufacturing losses and stopping manufacturing procedure inefficiencies has constantly been a steady battle for producers of all stripes. Today that is truer than ever, as developing calls for meets improved competition.


Ljubljana Reveals Its Secrets

#artificialintelligence

Originally published on Towards AI the World's Leading AI and Technology News and Media Company. If you are building an AI-related product or service, we invite you to consider becoming an AI sponsor. At Towards AI, we help scale AI and technology startups. Let us help you unleash your technology to the masses. Like most students, we faced the issue of finding the right apartment during our studies.


Council Post: Digital Transformation And Its Impact On Finance And The Banking System

#artificialintelligence

During this pandemic, we are becoming aware of the strong need for a massive digital transformation that allows the banking system to consciously and responsibly take care of customers, account holders, stockholders and other investing entities. Computerizing the processes in management and control has already passed. It's not enough in 2020 when a considerable part of savings and ordinary transactions are being acquired and processed by the new FinTech platforms and hybrid banks, often without holding a banking license. Some of the most powerful digital-only banks are Revolut, Chime, N26, Up, Monzo, Nubank and Starling. Many banks are exploiting digital tools, which is satisfying, but not enough until they move away from customer/user management as it happens in ordinary banks, face to face and hand to hand, and in private, corporate and retail sectors.


Why explainable AI is indispensable to Zillow's business

#artificialintelligence

Zillow, an online marketplace that facilitates the buying, selling, renting, financing, and remodeling of homes, employs lots of AI technologies to do things like estimate home prices. But the output of AI systems like these can be opaque, creating a "black box" problem where practitioners and customers can't audit the systems properly. Without transparency, serious problems like algorithmic bias can persist undetected, and trust in the models becomes impossible. For obvious ethical reasons, this is why explainable AI (XAI) is so crucial to the creation and deployment of AI systems, but pragmatically, it's also key to the success of AI-powered products and services from companies like Zillow. David Fagnan, director of applied science on the Zillow Offers team, discussed with VentureBeat how and why XAI is indispensable for the company.


Port Tampa Bay will test new security scanners. No stopping to empty pockets.

#artificialintelligence

Cruise passengers traveling out of Port Tampa Bay next spring could play a role in the testing of a new weapons detection system designed to improve safety in public places. Liberty Defense Holdings created HEXWAVE, which uses low-power, radar imaging and artificial intelligence to detect and identify weapons. The company has been developing the technology since 2018 and is ready to test it in 11 locations nationwide, CEO Bill Riker said. Similar to a metal detector, the system requires people to walk through portals one at a time as they are scanned. But unlike their more common counterpart, these scanners can detect both metal and non-metal items and use artificial intelligence to identify almost instantly what the item is and where it's located.