Goto

Collaborating Authors

 ice


Robust Transmission of Punctured Text with Large Language Model-based Recovery

Park, Sojeong, Noh, Hyeonho, Yang, Hyun Jong

arXiv.org Artificial Intelligence

With the recent advancements in deep learning, semantic communication which transmits only task-oriented features, has rapidly emerged. However, since feature extraction relies on learning-based models, its performance fundamentally depends on the training dataset or tasks. For practical scenarios, it is essential to design a model that demonstrates robust performance regardless of dataset or tasks. In this correspondence, we propose a novel text transmission model that selects and transmits only a few characters and recovers the missing characters at the receiver using a large language model (LLM). Additionally, we propose a novel importance character extractor (ICE), which selects transmitted characters to enhance LLM recovery performance. Simulations demonstrate that the proposed filter selection by ICE outperforms random filter selection, which selects transmitted characters randomly. Moreover, the proposed model exhibits robust performance across different datasets and tasks and outperforms traditional bit-based communication in low signal-to-noise ratio conditions.


Taming Knowledge Conflicts in Language Models

Li, Gaotang, Chen, Yuzhong, Tong, Hanghang

arXiv.org Artificial Intelligence

Language Models (LMs) often encounter knowledge conflicts when parametric memory contradicts contextual knowledge. Previous works attribute this conflict to the interplay between "memory heads" and "context heads", attention heads assumed to promote either memory or context exclusively. In this study, we go beyond this fundamental assumption by uncovering a critical phenomenon we term the "superposition of contextual information and parametric memory", where highly influential attention heads could simultaneously contribute to both memory and context. Building upon this insight, we propose Just Run Twice (JUICE), a test-time attention intervention method that steers LMs toward either parametric beliefs or contextual knowledge without requiring fine-tuning. JUICE identifies a set of reliable attention heads and leverages a dual-run approach to mitigate the superposition effects. Extensive experiments across 11 datasets and 6 model architectures demonstrate that JUICE sets the new state-of-the-art performance and robust generalization, achieving significant and consistent improvement across different domains under various conflict types. Finally, we theoretically analyze knowledge conflict and the superposition of contextual information and parametric memory in attention heads, which further elucidates the effectiveness of JUICE in these settings.


Senior Data Engineer at Deutsche Telekom IT Solutions Slovakia - Košice, Slovakia (Slovak Republic)

#artificialintelligence

Our brand Deutsche Telekom IT Solutions Slovakia entered the life of Košice region in 2006 under the name of T-Systems Slovakia and ever since has been inextricably linked with the region when became one of the founding members of Košice IT Valley. We have managed to grow from scratch to the second largest employer in the eastern part of the country with more than 3900 employees. Our goal is to proactively find new ways to improve and continuously transform into the type of company providing innovative information and communication technology services. We are seeking a highly skilled Senior Data Engineer to join our Common Data Intelligence HUB virtual team at T-Systems Germany. The Common Data Intelligence HUB is a team of approximately 100 engaged professionals dedicated to developing business intelligence solutions and open datasets for T-Systems.


Power BI Developer at Deutsche Telekom IT Solutions Slovakia - Košice, Slovakia (Slovak Republic)

#artificialintelligence

Our brand Deutsche Telekom IT Solutions Slovakia entered the life of Košice region in 2006 under the name of T-Systems Slovakia and ever since has been inextricably linked with the region when became one of the founding members of Košice IT Valley. We have managed to grow from scratch to the second largest employer in the eastern part of the country with more than 3900 employees. Our goal is to proactively find new ways to improve and continuously transform into the type of company providing innovative information and communication technology services. We are seeking a skilled and experienced Power BI Developer with an innovative mindset to join our Finance International Business Intelligence Team working in a Scrum environment. The ideal candidate will have the opportunity to work remotely from anywhere in Slovakia and will be responsible for designing and implementing data-driven solutions in a Microsoft BI cloud environment to support business intelligence needs across multiple foreign countries, with a focus on automation and implementing predictions, artificial intelligence and self-service BI in the future.


Few-Shot Anaphora Resolution in Scientific Protocols via Mixtures of In-Context Experts

Le, Nghia T., Bai, Fan, Ritter, Alan

arXiv.org Artificial Intelligence

Anaphora resolution is an important task for information extraction across a range of languages, text genres, and domains, motivating the need for methods that do not require large annotated datasets. In-context learning has emerged as a promising approach, yet there are a number of challenges in applying in-context learning to resolve anaphora. For example, encoding a single in-context demonstration that consists of: an anaphor, a paragraph-length context, and a list of corresponding antecedents, requires conditioning a language model on a long sequence of tokens, limiting the number of demonstrations per prompt. In this paper, we present MICE (Mixtures of In-Context Experts), which we demonstrate is effective for few-shot anaphora resolution in scientific protocols (Tamari et al., 2021). Given only a handful of training examples, MICE combines the predictions of hundreds of in-context experts, yielding a 30% increase in F1 score over a competitive prompt retrieval baseline. Furthermore, we show MICE can be used to train compact student models without sacrificing performance. As far as we are aware, this is the first work to present experimental results demonstrating the effectiveness of in-context learning on the task of few-shot anaphora resolution in scientific protocols.



ICE 'now operates as a domestic surveillance agency,' think tank says

Engadget

Although it's supposed to be restricted by surveillance rules at local, state and federal levels, Immigration and Customs Enforcement ( ICE) has built up a mass surveillance system that includes details on almost all US residents, according to a report from a major think tank. Researchers from Georgetown Law's Center on Privacy and Technology said ICE "now operates as a domestic surveillance agency" and that it was able to bypass regulations in part by purchasing databases from private companies. "Since its founding in 2003, ICE has not only been building its own capacity to use surveillance to carry out deportations but has also played a key role in the federal government's larger push to amass as much information as possible about all of our lives," the report's authors state. "By reaching into the digital records of state and local governments and buying databases with billions of data points from private companies, ICE has created a surveillance infrastructure that enables it to pull detailed dossiers on nearly anyone, seemingly at any time." The researchers spent two years looking into ICE to put together the extensive report, which is called "American Dragnet: Data-Driven Deportation in the 21st Century."


A Collective Aghastness

Slate

In the past month, workers in Silicon Valley have demanded that the large tech companies where they work stop doing business with federal agencies associated with the ghastlier policies of the Trump administration and local governments--and in some cases it's worked. Google said it would not renew a contract with the Pentagon to build an A.I. system for military drones after thousands of employees signed a petition and dozens quit in protest. Orlando, Florida's police department dropped Amazon's facial-recognition tech after a public outcry that included criticisms from Amazon employees opposed to the activity. Microsoft is keeping a contract with Immigration and Customs Enforcement despite demands from more than 100 of its employees who believe doing so signals a complicity with the administration's hard-line immigration policy. This activity has been facilitated by the Tech Workers Coalition, a volunteer group of professionals in the tech industry that has worked on a number of labor, justice, and equality issues in recent years.


Microsoft's ICE involvement illustrates tech's denial problem

Engadget

Nearly a decade ago, I had the good fortune of being one of the last people to interview the founder of Commodore International, Jack Tramiel (famous for Commodore computers and the popular C64), before he passed away. At 83 he died from heart failure after pioneering the consumer market for personal computers and home gaming, and working toward changing people's lives for the better through technology. What few people knew, and what I discovered in our interview, was that the foundational concept driving the Commodore 64 was Tramiel's vision for a future in which the Holocaust and its concentration camps (from which Jack survived but his father did not) would never be able to happen again. In our interview Mr. Tramiel told me: I made the market for the computer youth-driven. I went around the world meeting young people in computer clubs and showing them what the computer can do.


Amazon Employees To Jeff Bezos: Stop Selling Facial Recognition Tech To ICE

International Business Times

Amazon's operation has grown well beyond merely delivering items to people's homes. Jeff Bezos's massive corporation is now involved in everything from grocery shopping to fashion, but the recent revelation that Amazon technology assists law enforcement is a bridge too far for some employees. A group of Amazon employees (referred to as Amazonians) penned a letter to Bezos on Thursday asking the billionaire CEO to halt the sale of facial recognition technology to law enforcement agencies, The Hill reported. The software, called Amazon Web Services Rekognition, has been linked to government agencies like the controversial Immigration and Customs Enforcement, or ICE. The letter cited the United States government's history of injustice towards minorities in calling for Amazon to stop assisting ICE.