Goto

Collaborating Authors

 Antarctica


Can Large Language Models Address Open-Target Stance Detection?

arXiv.org Artificial Intelligence

Stance detection (SD) identifies a text's position towards a target, typically labeled as favor, against, or none. We introduce Open-Target Stance Detection (OTSD), the most realistic task where targets are neither seen during training nor provided as input. We evaluate Large Language Models (LLMs) GPT-4o, GPT-3.5, Llama-3, and Mistral, comparing their performance to the only existing work, Target-Stance Extraction (TSE), which benefits from predefined targets. Unlike TSE, OTSD removes the dependency of a predefined list, making target generation and evaluation more challenging. We also provide a metric for evaluating target quality that correlates well with human judgment. Our experiments reveal that LLMs outperform TSE in target generation when the real target is explicitly and not explicitly mentioned in the text. Likewise, for stance detection, LLMs excel in explicit cases with comparable performance in non-explicit in general.


Murder arrests after death of baby boy

BBC News

Two people have been arrested on suspicion of murder after a baby died in Stoke-on-Trent. Police and the ambulance service were called to Sherwin Road, Burslem, shortly after 09:00 BST on 27 August following the death of a baby boy. A woman, 26, and man, 25, were arrested on 30 August on suspicion of causing or allowing the death of a child. The pair, from Stoke-on-Trent, were further arrested on suspicion of murder on Tuesday, Staffordshire Police said. A spokesperson for the force said specialist officers were supporting the baby's family.


Enhancing LLM Problem Solving with REAP: Reflection, Explicit Problem Deconstruction, and Advanced Prompting

arXiv.org Artificial Intelligence

Large Language Models (LLMs) have transformed natural language processing, yet improving their problem-solving capabilities, particularly for complex, reasoning-intensive tasks, remains a persistent challenge. This paper introduces the REAP (Reflection, Explicit Problem Deconstruction, and Advanced Prompting) method, an innovative approach within the dynamic context generation framework. REAP guides LLMs through reflection on the query, deconstructing it into manageable components, and generating relevant context to enhance the solution process. We evaluated REAP using a dataset designed to expose LLM limitations, comparing zero-shot prompting with REAP-enhanced prompts across six state-of-the-art models: OpenAI's o1-preview, o1-mini, GPT-4o, GPT-4o-mini, Google's Gemini 1.5 Pro, and Claude 3.5 Sonnet. The results demonstrate notable performance gains, with o1-mini improving by 40.97%, GPT-4o by 66.26%, and GPT-4o-mini by 112.93%. Despite the already strong baseline performance of OpenAI's o1-preview, modest gains were observed. Beyond performance improvements, REAP offers a cost-effective solution; for example, GPT-4o-mini, which is approximately 100 times cheaper than o1-preview, delivered competitive results. REAP also improves the clarity of model outputs, making it easier for humans to understand the reasoning behind the results and simplifying the process of identifying and addressing any issues. These findings demonstrate REAP's potential to greatly improve the capabilities of LLMs, providing both better performance and increased cost-efficiency across a wide range of applications.


Teenager invents robot to solve Rubik's Cube

BBC News

Teenager invents robot to solve Rubik's Cube BBCRuarcc the year 10 student who has programmed a robot that can solve a Rubik's Cube puzzle A 13-year-old schoolboy has invented a Lego robot that can solve a Rubik's cube. Ruarcc, from St Malachy's College in north Belfast, first took steps to create puzzle-solving robot prototypes in his second year at school, aged 12. This was made possible after the school launched its creative digital technology hub (CDTH) last year. Teacher Clare McGrath commented she "didn't believe" that Ruarcc's robot would work at first.'People are amazed my robot can solve Rubik's Cube' Ruarcc told BBC News NI it was "frustrating", but he worked on making it better. "People tend to be amazed that it can solve one," he said.


Wasserstein Distributionally Robust Multiclass Support Vector Machine

arXiv.org Machine Learning

We study the problem of multiclass classification for settings where data features $\mathbf{x}$ and their labels $\mathbf{y}$ are uncertain. We identify that distributionally robust one-vs-all (OVA) classifiers often struggle in settings with imbalanced data. To address this issue, we use Wasserstein distributionally robust optimization to develop a robust version of the multiclass support vector machine (SVM) characterized by the Crammer-Singer (CS) loss. First, we prove that the CS loss is bounded from above by a Lipschitz continuous function for all $\mathbf{x} \in \mathcal{X}$ and $\mathbf{y} \in \mathcal{Y}$, then we exploit strong duality results to express the dual of the worst-case risk problem, and we show that the worst-case risk minimization problem admits a tractable convex reformulation due to the regularity of the CS loss. Moreover, we develop a kernel version of our proposed model to account for nonlinear class separation, and we show that it admits a tractable convex upper bound. We also propose a projected subgradient method algorithm for a special case of our proposed linear model to improve scalability. Our numerical experiments demonstrate that our model outperforms state-of-the art OVA models in settings where the training data is highly imbalanced. We also show through experiments on popular real-world datasets that our proposed model often outperforms its regularized counterpart as the first accounts for uncertain labels unlike the latter.


John Lewis brings back 'never knowingly undersold'

BBC News

John Lewis brings back'never knowingly undersold' Getty Images Retailer John Lewis is bringing back its "never knowingly undersold" price pledge from Monday, two years after abandoning it. It will also apply to online sales for the first time, whereas it previously only applied to in-store shopping, and will use AI to match the prices of 25 top retailers. The department store chain has been trying to win back customers after a tough few years that has seen it cut jobs and close several stores. It swung back to profit earlier this year, but is expected to continue shedding jobs as it seeks to revive its fortunes. The decision by John Lewis' new managing director Pete Ruis to restore the price pledge marks a change of direction from his predecessor.


LM-PUB-QUIZ: A Comprehensive Framework for Zero-Shot Evaluation of Relational Knowledge in Language Models

arXiv.org Artificial Intelligence

Knowledge probing evaluates the extent to which a language model (LM) has acquired relational knowledge during its pre-training phase. It provides a cost-effective means of comparing LMs of different sizes and training setups and is useful for monitoring knowledge gained or lost during continual learning (CL). In prior work, we presented an improved knowledge probe called BEAR (Wiland et al., 2024), which enables the comparison of LMs trained with different pre-training objectives (causal and masked LMs) and addresses issues of skewed distributions in previous probes to deliver a more unbiased reading of LM knowledge. With this paper, we present LM-PUB- QUIZ, a Python framework and leaderboard built around the BEAR probing mechanism that enables researchers and practitioners to apply it in their work. It provides options for standalone evaluation and direct integration into the widely-used training pipeline of the Hugging Face TRANSFORMERS library. Further, it provides a fine-grained analysis of different knowledge types to assist users in better understanding the knowledge in each evaluated LM. We publicly release LM-PUB-QUIZ as an open-source project.


Putting a Fine-Art Touch on Fixer-Uppers

The New Yorker

You can view art (e.g., at a museum). "To go from photographing the getaway to making the getaway, it's like art becoming reality," the photographer Gray Malin said the other day. He was sitting behind the wheel of a blue Range Rover, dressed in a denim shirt, white jeans, and raffia loafers, cruising up the 101 to his latest work: a house he has renovated in Montecito, California, in order to rent to visitors. "How I vacation is how I want you to vacation," he said. As an artist, Malin specializes in glossy portraits of the good life: beaches, boats, a plane landing in St. Bart's.


Seven things we learned from Gamescom opening night

BBC News

It has been a year with no major new console launches and where the industry has seen strikes and cuts with thousands of workers being laid off. The opening night of Gamescom is often an opportunity for a big shiny night to get fans all excited for the year ahead. Setting the stage for the next 12 months, here are the biggest things we found out from Europe's biggest gaming show in Germany. In a year when games became films, and films became games, the convention centre in Cologne saw a night all about the big trailers. This year, Borderlands has taken attention for its movie adaptation starring Cate Blanchett and Kevin Hart. That film received some of the year's harshest reviews, but that has not scuppered plans for a new game in the mainline series.


Discover-then-Name: Task-Agnostic Concept Bottlenecks via Automated Concept Discovery

arXiv.org Artificial Intelligence

Concept Bottleneck Models (CBMs) have recently been proposed to address the 'black-box' problem of deep neural networks, by first mapping images to a human-understandable concept space and then linearly combining concepts for classification. Such models typically require first coming up with a set of concepts relevant to the task and then aligning the representations of a feature extractor to map to these concepts. However, even with powerful foundational feature extractors like CLIP, there are no guarantees that the specified concepts are detectable. In this work, we leverage recent advances in mechanistic interpretability and propose a novel CBM approach -- called Discover-then-Name-CBM (DN-CBM) -- that inverts the typical paradigm: instead of pre-selecting concepts based on the downstream classification task, we use sparse autoencoders to first discover concepts learnt by the model, and then name them and train linear probes for classification. Our concept extraction strategy is efficient, since it is agnostic to the downstream task, and uses concepts already known to the model. We perform a comprehensive evaluation across multiple datasets and CLIP architectures and show that our method yields semantically meaningful concepts, assigns appropriate names to them that make them easy to interpret, and yields performant and interpretable CBMs. Code available at https://github.com/neuroexplicit-saar/discover-then-name.