Personal
'CHiPs' star Erik Estrada says certain people using AI are not 'very Christian'
"CHiPs" star Erik Estrada shared a warning about how artificial intelligence can "destroy lives." During an interview with Fox News Digital, the 75-year-old actor and "Divine Renovation" host acknowledged the benefits of AI but cautioned that the new technology is also frequently being used for nefarious purposes. "I think just like the Internet, just like the cell phones, just like everything -- they need to just use the positive side of it," Estrada said. "The side which can help or employ and create goodwill, good things, good jobs, good fortune for people that want to go in that direction and not, of course, use the negative stuff." "CHiPs" star Erik Estrada warned about the dangers posed by AI. (Brian To/FilmMagic) Estrada pointed to how AI can be used to create deepfakes -- deceptive pictures, videos and audio that misrepresent people or events.
Democrat senator targeted by deepfake impersonator of Ukrainian official on Zoom call: reports
An Ohio-based company sells robotic dogs being used by the Ukrainian military against Russia, which have the ability to be outfitted with flamethrowers. Authorities are investigating a mysterious "deep fake" video call that successfully impersonated a Ukrainian high official. Democratic Sen. Benjamin Cardin announced Wednesday that he had turned over materials to law enforcement after an unknown suspect had tricked him onto a video call via impersonating a foreign official. "In recent days, a malign actor engaged in a deceptive attempt to have a conversation with me by posing as a known individual. After immediately becoming clear that the individual I was engaging with was not who they claimed to be, I ended the call and my office took swift action, alerting the relevant authorities."
The Download: a CRISPR patent battle, and the promise of tiny AI
In the decade-long fight to control CRISPR, the super-tool for modifying DNA, it's been common for lawyers to try to overturn patents held by competitors. But now, in a surprise twist, the team that earned the Nobel Prize in chemistry for developing CRISPR is asking to cancel two of their own seminal patents, MIT Technology Review has learned. The request to withdraw the pair of European patents, by lawyers for Emmanuelle Charpentier and Jennifer Doudna, comes after a damaging August opinion from a European technical appeals board, which ruled that the duo's earliest patent filing didn't explain CRISPR well enough for other scientists to use it and doesn't count as a proper invention. The decision could have major ramifications regarding who gets to collect the lucrative licensing fees on using the technology.Read the full story. What's new: The Allen Institute for Artificial Intelligence (Ai2), a research nonprofit, is releasing a family of open-source multimodal language models, called Molmo, that it says perform as well as top proprietary models from OpenAI, Google, and Anthropic.
AIhub monthly digest: September 2024 – real-time payments, evaluating dataset diversity, and AfriClimate AI at the Deep Learning Indaba
Welcome to our monthly digest, where you can catch up with any AIhub stories you may have missed, peruse the latest news, recap recent events, and more. This month, we learn about a framework to evaluate diversity in datasets, find out how banks may strategically mitigate their risk from fraud in real-time payment systems, and hear about the AfriClimate AI workshop at the Deep Learning Indaba. Don't Just Claim It, Jerone Andrews and colleagues propose using measurement theory from the social sciences as a framework to improve the collection and evaluation of diverse machine learning datasets. We spoke to Jerone about this work, which won a best paper award at ICML 2024. Real-time payments offer a fast processing time (of around 10 seconds), allowing for near-immediate receipt of funds.
RoleBreak: Character Hallucination as a Jailbreak Attack in Role-Playing Systems
Tang, Yihong, Wang, Bo, Wang, Xu, Zhao, Dongming, Liu, Jing, Zhang, Jijun, He, Ruifang, Hou, Yuexian
Role-playing systems powered by large language models (LLMs) have become increasingly influential in emotional communication applications. However, these systems are susceptible to character hallucinations, where the model deviates from predefined character roles and generates responses that are inconsistent with the intended persona. This paper presents the first systematic analysis of character hallucination from an attack perspective, introducing the RoleBreak framework. Our framework identifies two core mechanisms-query sparsity and role-query conflict-as key factors driving character hallucination. Leveraging these insights, we construct a novel dataset, RoleBreakEval, to evaluate existing hallucination mitigation techniques. Our experiments reveal that even enhanced models trained to minimize hallucination remain vulnerable to attacks. To address these vulnerabilities, we propose a novel defence strategy, the Narrator Mode, which generates supplemental context through narration to mitigate role-query conflicts and improve query generalization. Experimental results demonstrate that Narrator Mode significantly outperforms traditional refusal-based strategies by reducing hallucinations, enhancing fidelity to character roles and queries, and improving overall narrative coherence.
LLM Echo Chamber: personalized and automated disinformation
Recent advancements have showcased the capabilities of Large Language Models like GPT4 and Llama2 in tasks such as summarization, translation, and content review. However, their widespread use raises concerns, particularly around the potential for LLMs to spread persuasive, humanlike misinformation at scale, which could significantly influence public opinion. This study examines these risks, focusing on LLMs ability to propagate misinformation as factual. To investigate this, we built the LLM Echo Chamber, a controlled digital environment simulating social media chatrooms, where misinformation often spreads. Echo chambers, where individuals only interact with like minded people, further entrench beliefs. By studying malicious bots spreading misinformation in this environment, we can better understand this phenomenon. We reviewed current LLMs, explored misinformation risks, and applied sota finetuning techniques. Using Microsoft phi2 model, finetuned with our custom dataset, we generated harmful content to create the Echo Chamber. This setup, evaluated by GPT4 for persuasiveness and harmfulness, sheds light on the ethical concerns surrounding LLMs and emphasizes the need for stronger safeguards against misinformation.
InsightPulse: An IoT-based System for User Experience Interview Analysis
Lyu, Dian, Lu, Yuetong, He, Jassie, Abrar, Murad Mehrab, Xie, Ruijun, Raiti, John
Conducting efficient and effective user experience (UX) interviews often poses challenges, such as maintaining focus on key topics and managing the duration of interviews and post-interview analyses. To address these issues, this paper introduces InsightPulse, an Internet of Things (IoT)-based hardware and software system designed to streamline and enhance the UX interview process through speech analysis and Artificial Intelligence. InsightPulse provides real-time support during user interviews by automatically identifying and highlighting key discussion points, proactively suggesting follow-up questions, and generating thematic summaries. These features enable more insightful discoveries and help to manage interview duration effectively. Additionally, the system features a robust backend analytics dashboard that simplifies the post-interview review process, thus facilitating the quick extraction of actionable insights and enhancing overall UX research efficiency.
Learning When to Retrieve, What to Rewrite, and How to Respond in Conversational QA
Roy, Nirmal, Ribeiro, Leonardo F. R., Blloshmi, Rexhina, Small, Kevin
Augmenting Large Language Models (LLMs) with information retrieval capabilities (i.e., Retrieval-Augmented Generation (RAG)) has proven beneficial for knowledge-intensive tasks. However, understanding users' contextual search intent when generating responses is an understudied topic for conversational question answering (QA). This conversational extension leads to additional concerns when compared to single-turn QA as it is more challenging for systems to comprehend conversational context and manage retrieved passages over multiple turns. In this work, we propose a method for enabling LLMs to decide when to retrieve in RAG settings given a conversational context. When retrieval is deemed necessary, the LLM then rewrites the conversation for passage retrieval and judges the relevance of returned passages before response generation. Operationally, we build on the single-turn SELF-RAG framework (Asai et al., 2023) and propose SELF-multi-RAG for conversational settings. SELF-multi-RAG demonstrates improved capabilities over single-turn variants with respect to retrieving relevant passages (by using summarized conversational context) and assessing the quality of generated responses. Experiments on three conversational QA datasets validate the enhanced response generation capabilities of SELF-multi-RAG, with improvements of ~13% measured by human annotation.
On a measure of intelligence
The measure of intelligence is the ability to change. Abstract The Fall 2024 Logic in Computer Science column of the Bulletin of EATCS is a little discussion on intelligence, measuring intelligence, and related issues, provoked by a fascinating must-read article "On the measure of intelligence" by François Chollet. The discussion includes a modicum of critique of the article. Q: Is it about psychology? Chollet is a prominent figure in AI. Q: We spoke about AI last spring. But you didn't seem to be interested in AI before that. A: This is largely correct, though I read Norbert Wiener's "Cybernetics" [18], when it was translated to Russian in 1968, and was taken with it. For a while I tried to follow cybernetics developments, at least in the USSR.
On logic and generative AI
Gurevich, Yuri, Blass, Andreas
This article was originally written for the June 2024 issue of the Bulletin of European Association for Theoretical Computer Science, in the framework of the "Logic in Computer Science" column administered by Yuri Gurevich. In the following pages, the article is reproduced as is. The ongoing AI revolution raises many foundational problems. For quite a while, I felt that the issue needs to be addressed in this column. Not being an AI expert, I was looking for volunteers. This didn't work, and so one day I took a deep breath and started to write an article myself. Andreas Blass, my long-time collaborator, was reluctant to join me, but eventually he agreed. A hundred years ago, logic was almost synonymous with foundational studies. I tried to rekindle that tradition in [5]. The goal of the following dialog is to provoke young logicians with a taste for foundations to notice the foundational problems raised by the ongoing AI revolution. I think the most beautiful thing about deep learning is that it actually works. Q: I just learned that Daniel Kahneman, Nobel laureate in economics and the author of "Thinking, fast and slow" [7], passed away on March 27, 2024. I heard a lot about this book but have never read it.