ice
Taming Knowledge Conflicts in Language Models
Li, Gaotang, Chen, Yuzhong, Tong, Hanghang
Language Models (LMs) often encounter knowledge conflicts when parametric memory contradicts contextual knowledge. Previous works attribute this conflict to the interplay between "memory heads" and "context heads", attention heads assumed to promote either memory or context exclusively. In this study, we go beyond this fundamental assumption by uncovering a critical phenomenon we term the "superposition of contextual information and parametric memory", where highly influential attention heads could simultaneously contribute to both memory and context. Building upon this insight, we propose Just Run Twice (JUICE), a test-time attention intervention method that steers LMs toward either parametric beliefs or contextual knowledge without requiring fine-tuning. JUICE identifies a set of reliable attention heads and leverages a dual-run approach to mitigate the superposition effects. Extensive experiments across 11 datasets and 6 model architectures demonstrate that JUICE sets the new state-of-the-art performance and robust generalization, achieving significant and consistent improvement across different domains under various conflict types. Finally, we theoretically analyze knowledge conflict and the superposition of contextual information and parametric memory in attention heads, which further elucidates the effectiveness of JUICE in these settings.
- Asia > China (0.14)
- North America > United States > Illinois (0.14)
- Information Technology > Artificial Intelligence > Representation & Reasoning (1.00)
- Information Technology > Artificial Intelligence > Natural Language > Large Language Model (1.00)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning (0.92)
- Information Technology > Artificial Intelligence > Natural Language > Chatbot (0.67)
Senior Data Engineer at Deutsche Telekom IT Solutions Slovakia - Košice, Slovakia (Slovak Republic)
Our brand Deutsche Telekom IT Solutions Slovakia entered the life of Košice region in 2006 under the name of T-Systems Slovakia and ever since has been inextricably linked with the region when became one of the founding members of Košice IT Valley. We have managed to grow from scratch to the second largest employer in the eastern part of the country with more than 3900 employees. Our goal is to proactively find new ways to improve and continuously transform into the type of company providing innovative information and communication technology services. We are seeking a highly skilled Senior Data Engineer to join our Common Data Intelligence HUB virtual team at T-Systems Germany. The Common Data Intelligence HUB is a team of approximately 100 engaged professionals dedicated to developing business intelligence solutions and open datasets for T-Systems.
Power BI Developer at Deutsche Telekom IT Solutions Slovakia - Košice, Slovakia (Slovak Republic)
Our brand Deutsche Telekom IT Solutions Slovakia entered the life of Košice region in 2006 under the name of T-Systems Slovakia and ever since has been inextricably linked with the region when became one of the founding members of Košice IT Valley. We have managed to grow from scratch to the second largest employer in the eastern part of the country with more than 3900 employees. Our goal is to proactively find new ways to improve and continuously transform into the type of company providing innovative information and communication technology services. We are seeking a skilled and experienced Power BI Developer with an innovative mindset to join our Finance International Business Intelligence Team working in a Scrum environment. The ideal candidate will have the opportunity to work remotely from anywhere in Slovakia and will be responsible for designing and implementing data-driven solutions in a Microsoft BI cloud environment to support business intelligence needs across multiple foreign countries, with a focus on automation and implementing predictions, artificial intelligence and self-service BI in the future.
- Information Technology > Artificial Intelligence (0.72)
- Information Technology > Software (0.63)
- Information Technology > Data Science (0.61)
- (2 more...)
ICE 'now operates as a domestic surveillance agency,' think tank says
Although it's supposed to be restricted by surveillance rules at local, state and federal levels, Immigration and Customs Enforcement ( ICE) has built up a mass surveillance system that includes details on almost all US residents, according to a report from a major think tank. Researchers from Georgetown Law's Center on Privacy and Technology said ICE "now operates as a domestic surveillance agency" and that it was able to bypass regulations in part by purchasing databases from private companies. "Since its founding in 2003, ICE has not only been building its own capacity to use surveillance to carry out deportations but has also played a key role in the federal government's larger push to amass as much information as possible about all of our lives," the report's authors state. "By reaching into the digital records of state and local governments and buying databases with billions of data points from private companies, ICE has created a surveillance infrastructure that enables it to pull detailed dossiers on nearly anyone, seemingly at any time." The researchers spent two years looking into ICE to put together the extensive report, which is called "American Dragnet: Data-Driven Deportation in the 21st Century."
Tech Employees Are Rallying Against Their Companies' Work With ICE
On June 17th, Salesforce CEO Marc Benioff retweeted images from a CBS reporter of a detention facility in McAllen, Texas where children slept on the floor, covered in emergency blankets. Just a few days later, Microsoft CEO Satya Nadella sent an all-staff email calling President Donald Trump's immigration policy "cruel and abusive." In March, Beinoff's company signed a contract with US Customs and Border Patrol (CBP) to provide cloud services in order to boost the agency's hiring efforts, something that CBP has long struggled with. And at Microsoft, employees have organized to demand that Microsoft cancel its $19.4 million contract with Immigration and Customs Enforcement (ICE) for data processing and, potentially, facial recognition software. An NBC investigation also identified active contracts between ICE and a host of prominent tech companies--Hewlett Packard Enterprise, Thomson Reuters, Motorola Solutions, and Palantir--each worth tens of millions.
- Information Technology (1.00)
- Government > Regional Government > North America Government > United States Government (1.00)
- Government > Immigration & Customs (1.00)
Microsoft employees criticize the firm's contracts with ICE
Microsoft is the latest tech giant to find itself in the crosshairs over a controversial government contract. In January, it was announced that Microsoft's Azure Government arm was working with U.S. Immigration and Customs Enforcement (ICE) to assist with'facial recognition and identification', providing access to its deep learning AI technology. Along with that, Microsoft said it was'proud to support' ICE - a section that was deleted and, ultimately, restored after it was made public, according to BuzzFeed. In January, it was announced that Microsoft's Azure Government arm was working with ICE to assist with'facial recognition and identification,' providing access to its deep learning AI tech That hasn't stopped many from calling on Microsoft employees to resign from the company or urge CEO Satya Nadella to speak about the firm's dealings with ICE. Microsoft has since responded that deleting the content was a'mistake' and issued a follow up blog post further detailing the matter. 'In response to questions we want to be clear: Microsoft is not working with U.S. Immigration and Customs Enforcement or U.S. Customs and Border Protection on any projects related to separating children from their families at the border, and contrary to some speculation, we are not aware of Azure or Azure services being used for this purpose,' the firm wrote in a statement.
- North America > United States > Florida > Orange County > Orlando (0.16)
- Asia > China (0.06)
- North America > United States > Oregon (0.05)
A New Basis for Spreadsheet Computing
There is a fundamental mismatch between the computational basis of spreadsheets and our knowledge of the real world. In spreadsheets, numeric data are represented as exact numbers and their mutual relations as functions, whose values (output) are computed from given argument values (input). However, in the real world, data are often inexact and uncertain in many ways, and the relationships, that is, constraints, between input and output are far more complicated. This article shows that interval constraint solving, an emerging AIbased technology, provides a more versatile and useful foundation for spreadsheets. The new computational basis is 100-percent downward compatible with the traditional spreadsheet paradigm.
ICE: An Expressive Iterative Combinatorial Exchange
Lubin, B., Juda, A. I., Cavallo, R., Lahaie, S., Shneidman, J., Parkes, D. C.
We present the design and analysis of the first fully expressive, iterative combinatorial exchange (ICE). The exchange incorporates a tree-based bidding language (TBBL) that is concise and expressive for CEs. Bidders specify lower and upper bounds in TBBL on their value for different trades and refine these bounds across rounds. These bounds allow price discovery and useful preference elicitation in early rounds, and allow termination with an efficient trade despite partial information on bidder valuations. All computation in the exchange is carefully optimized to exploit the structure of the bid-trees and to avoid enumerating trades. A proxied interpretation of a revealed-preference activity rule, coupled with simple linear prices, ensures progress across rounds. The exchange is fully implemented, and we give results demonstrating several aspects of its scalability and economic properties with simulated bidding strategies.
- Europe > United Kingdom > England > Cambridgeshire > Cambridge (0.04)
- North America > United States > New York > New York County > New York City (0.04)
- North America > United States > Massachusetts > Middlesex County > Cambridge (0.04)
- North America > United States > Hawaii (0.04)
- Banking & Finance (0.67)
- Transportation > Air (0.46)