Media
Last Layer Logits to Logic: Empowering LLMs with Logic-Consistent Structured Knowledge Reasoning
Li, Songze, Liu, Zhiqiang, Gong, Zhaoyan, Guo, Xiaoke, Gui, Zhengke, Chen, Huajun, Zhang, Wen
Large Language Models (LLMs) achieve excellent performance in natural language reasoning tasks through pre-training on vast unstructured text, enabling them to understand the logic in natural language and generate logic-consistent responses. However, the representational differences between unstructured and structured knowledge make LLMs inherently struggle to maintain logic consistency, leading to \textit{Logic Drift} challenges in structured knowledge reasoning tasks such as Knowledge Graph Question Answering (KGQA). Existing methods address this limitation by designing complex workflows embedded in prompts to guide LLM reasoning. Nevertheless, these approaches only provide input-level guidance and fail to fundamentally address the \textit{Logic Drift} in LLM outputs. Additionally, their inflexible reasoning workflows cannot adapt to different tasks and knowledge graphs. To enhance LLMs' logic consistency in structured knowledge reasoning, we specifically target the logits output from the autoregressive generation process. We propose the \textit{Logits-to-Logic} framework, which incorporates logits strengthening and logits filtering as core modules to correct logical defects in LLM outputs. Extensive experiments show that our approach significantly improves LLMs' logic consistency in structured knowledge reasoning and achieves state-of-the-art performance on multiple KGQA benchmarks.
Matthew McConaughey and Michael Caine sign voice deal with AI company
Oscar-winning actors Matthew McConaughey and Michael Caine have both signed a deal with the AI audio company ElevenLabs. The New York-based company can now create AI-generated versions of their voices as part of a bid to solve a "key ethical challenge" in the artificial intelligence industry's alliance with Hollywood. Emma Thompson speaks of her'intense irritation' with AI McConaughey, who has also invested in the company and collaborated with it since 2022, will now allow ElevenLabs to translate his newsletter, Lyrics of Livin', into a Spanish-language audio version using his voice. In a statement, the Dallas Buyers Club actor said he was "impressed" by ElevenLabs and wanted the partnership to help him "reach and connect with even more people". ElevenLabs is also launching the Iconic Voices Marketplace, which will allow brands to partner with the company and use officially licensed celebrity voices for AI-generated usage.
Trafficked turtles get a second chance at life in New Jersey sanctuary
The Turtle Conservancy is rescuing reptiles, while preserving their precious DNA. Breakthroughs, discoveries, and DIY tips sent every weekday. Nestled in rural New Jersey's rolling hills, a top secret animal sanctuary is keeping its occupants safe. The oasis is not for the official state animal (the horse) or even birds rescued from the Jersey shore. This hidden refuge belongs to turtles and tortoises.
Britain's ambitious plan to end animal testing: UK government reveals shift away from 'suffering' creatures toward futuristic organ-on-a-chip systems, AI, and 3D-printed tissues
Trump boasts Republicans'broke' Chuck Schumer during shutdown slugfest amid growing calls from the far-left to remove minority leader Camp Mystic accused of neglect in'self-created disaster' after 25 young campers die in flood Gavin Newsom just let slip the Democrats' dirty secret that could totally destroy them... and Trump is loving it: MARK HALPERIN You think you look so youthful and trendy. But these are the brutal facts about your shoes, hats, jumpers and denim. You're being lied to - this is the truth I'm a heart surgeon... here is what you must NEVER do after turning 40 Paris Jackson reveals she has a HOLE in her nose from drug abuse... after celebrating five years of sobriety Warren Buffett's final farewell: the billionaire who once fingerprinted nuns says he's'going quiet' Shamed Allison Mack's husband revealed as former neo-Nazi Kim Kardashian flaunts her famous curves in daring cut-out gown at All's Fair premiere in Brazil as she's seen for first time since failing the bar exam Timothee Chalamet has DUMPED Kylie Jenner, insiders claim... and reveal brutal new way he is humiliating her Trump dismisses economic anxiety with'fake' polls and reveals reason 600,000 Chinese students are in US Was ANOTHER person inside the house with'Foxy Knoxy' when Meredith Kercher was killed 18 years ago? That's the sensational new claim of the Italian prosecutor who put the American student in jail, DAVID JONES reveals Disgraceful lies of'everyday' mom exposed after she chased down ICE agents in her Ford Mustang then cried wolf Millennial socialist who still lives off parents' money is set to become next Seattle mayor Netflix's historical drama Death By Lightning branded woke after fans spot bizarre detail... can you see it? Revealed: MAGA Trump-supporting influencer and friend are subject of police photo appeal over alleged'assault and racial abuse' in London AMANDA PLATELL: It's time the royals stripped Beatrice and Eugenie of their titles.
Gen Alpha is cancelling the KEYBOARD: Youngsters won't ever have to write emails when they join the workforce - and will send voice notes to their boss instead, report reveals
Trump boasts Republicans'broke' Chuck Schumer during shutdown slugfest amid growing calls from the far-left to remove minority leader Camp Mystic accused of neglect in'self-created disaster' after 25 young campers die in flood Gavin Newsom just let slip the Democrats' dirty secret that could totally destroy them... and Trump is loving it: MARK HALPERIN You think you look so youthful and trendy. But these are the brutal facts about your shoes, hats, jumpers and denim. You're being lied to - this is the truth I'm a heart surgeon... here is what you must NEVER do after turning 40 Paris Jackson reveals she has a HOLE in her nose from drug abuse... after celebrating five years of sobriety Warren Buffett's final farewell: the billionaire who once fingerprinted nuns says he's'going quiet' Shamed Allison Mack's husband revealed as former neo-Nazi Kim Kardashian flaunts her famous curves in daring cut-out gown at All's Fair premiere in Brazil as she's seen for first time since failing the bar exam Timothee Chalamet has DUMPED Kylie Jenner, insiders claim... and reveal brutal new way he is humiliating her Trump dismisses economic anxiety with'fake' polls and reveals reason 600,000 Chinese students are in US Was ANOTHER person inside the house with'Foxy Knoxy' when Meredith Kercher was killed 18 years ago? That's the sensational new claim of the Italian prosecutor who put the American student in jail, DAVID JONES reveals Disgraceful lies of'everyday' mom exposed after she chased down ICE agents in her Ford Mustang then cried wolf'Grand conspiracy' probe seeks to prove cabal of'deep state' Democrats led by Obama tried to destroy Trump Taylor Swift's bridesmaids revealed as A-listers are tapped to join her wedding squad ahead of Travis Kelce nuptials Millennial socialist who still lives off parents' money is set to become next Seattle mayor Netflix's historical drama Death By Lightning branded woke after fans spot bizarre detail... can you see it? New York City's tallest men reveal the downsides to being over 7ft... from brutal injuries to dating struggles Gen Alpha is cancelling the KEYBOARD: Youngsters won't ever have to write emails when they join the workforce - and will send voice notes to their boss instead, report reveals READ MORE: Microsoft's paperclip mascot Clippy delights users as it returns From floppy disks to fax machines, many once-common office technologies would be baffling to the younger generation.
Artist sneaks AI-generated print into museum gallery
An artist sneaked an AI-generated print on to a gallery wall before bemused visitors alerted museum staff. The print was hung up at National Museum Cardiff by secretive artist Elias Marrow, who said his Empty Plate piece - depicting a young boy in school uniform holding a plate - was viewed by a few hundred people before it was removed. One visitor who noticed the artwork asked a member of staff about it, but said the museum worker admitted they had no idea about the piece or when it arrived. An Amgueddfa Cymru spokesperson said: An item was placed without permission on a gallery wall in National Museum Cardiff. We were alerted to this and have removed the item in question.
Multiple Streams of Knowledge Retrieval: Enriching and Recalling in Transformers
Nief, Todd, Reber, David, Richardson, Sean, Holtzman, Ari
When an LLM learns a new fact during finetuning (e.g., new movie releases, newly elected pope, etc.), where does this information go? Are entities enriched with relation information, or do models recall information just-in-time before a prediction? Or, are ``all of the above'' true with LLMs implementing multiple redundant heuristics? Existing localization approaches (e.g., activation patching) are ill-suited for this analysis because they usually \textit{replace} parts of the residual stream, thus overriding previous information. To fill this gap, we propose \emph{dynamic weight grafting}, a technique that selectively grafts weights from a finetuned model onto a pretrained model. Using this technique, we show two separate pathways for retrieving finetuned relation information: 1) ``enriching" the residual stream with relation information while processing the tokens that correspond to an entity (e.g., ``Zendaya'' in ``Zendaya co-starred with John David Washington'') and 2) ``recalling" this information at the final token position before generating a target fact. In some cases, models need information from both of these pathways to correctly generate finetuned facts while, in other cases, either the ``enrichment" or ``recall" pathway alone is sufficient. We localize the ``recall'' pathway to model components -- finding that ``recall" occurs via both task-specific attention mechanisms and an entity-specific extraction step in the feedforward networks of the final layers before the target prediction. By targeting model components and parameters, as opposed to just activations, we are able to understand the \textit{mechanisms} by which finetuned knowledge is retrieved during generation.