Personal
Hodgkinson targets 800m world record set in 1983
Olympic 1500m bronze medal winner Georgia Bell said she is still undecided about whether to become a full-time athlete. The 30-year-old only returned to running three years ago having fallen out of love with the sport. Bell still works for a a cyber security software company in London. "I've been on a break over the summer to focus on the Olympics and the plan is to go back in September," she said. "Work have been super-supportive and we'll see what happens. I think it will be really difficult to balance both. So it's something I'm going to think about."
TimeSense: Multi-Person Device-free Indoor Localization via RTT
Mohsen, Mohamed, Rizk, Hamada, Yamaguch, Hirozumi, Youssef, Moustafa
Locating the persons moving through an environment without the necessity of them being equipped with special devices has become vital for many applications including security, IoT, healthcare, etc. Existing device-free indoor localization systems commonly rely on the utilization of Received Signal Strength Indicator (RSSI) and WiFi Channel State Information (CSI) techniques. However, the accuracy of RSSI is adversely affected by environmental factors like multi-path interference and fading. Additionally, the lack of standardization in CSI necessitates the use of specialized hardware and software. In this paper, we present TimeSense, a deep learning-based multi-person device-free indoor localization system that addresses these challenges. TimeSense leverages Time of Flight information acquired by the fine-time measurement protocol of IEEE 802.11-2016 standard. Specifically, the measured round trip time between the transmitter and receiver is influenced by the dynamic changes in the environment induced by human presence. TimeSense effectively detects this anomalous behavior using a stacked denoising auto-encoder model, thereby estimating the user's location. The system incorporates a probabilistic approach on top of the deep learning model to ensure seamless tracking of the users. The evaluation of TimeSene in two realistic environments demonstrates its efficacy, achieving a median localization accuracy of 1.57 and 2.65 meters. This surpasses the performance of state-of-the-art techniques by 49% and 103% in the two testbeds.
VERA: Validation and Evaluation of Retrieval-Augmented Systems
Ding, Tianyu, Banerjee, Adi, Mombaerts, Laurent, Li, Yunhong, Borogovac, Tarik, Weinstein, Juan Pablo De la Cruz
The increasing use of Retrieval-Augmented Generation (RAG) systems in various applications necessitates stringent protocols to ensure RAG systems accuracy, safety, and alignment with user intentions. In this paper, we introduce VERA (Validation and Evaluation of Retrieval-Augmented Systems), a framework designed to enhance the transparency and reliability of outputs from large language models (LLMs) that utilize retrieved information. VERA improves the way we evaluate RAG systems in two important ways: (1) it introduces a cross-encoder based mechanism that encompasses a set of multidimensional metrics into a single comprehensive ranking score, addressing the challenge of prioritizing individual metrics, and (2) it employs Bootstrap statistics on LLM-based metrics across the document repository to establish confidence bounds, ensuring the repositorys topical coverage and improving the overall reliability of retrieval systems. Through several use cases, we demonstrate how VERA can strengthen decision-making processes and trust in AI applications. Our findings not only contribute to the theoretical understanding of LLM-based RAG evaluation metric but also promote the practical implementation of responsible AI systems, marking a significant advancement in the development of reliable and transparent generative AI technologies.
Order Matters in Hallucination: Reasoning Order as Benchmark and Reflexive Prompting for Large-Language-Models
Large language models (LLMs) have generated significant attention since their inception, finding applications across various academic and industrial domains. However, these models often suffer from the "hallucination problem", where outputs, though grammatically and logically coherent, lack factual accuracy or are entirely fabricated. A particularly troubling issue discovered and widely discussed recently is the numerical comparison error where multiple LLMs incorrectly infer that "9.11$>$9.9". We discovered that the order in which LLMs generate answers and reasoning impacts their consistency. Specifically, results vary significantly when an LLM generates an answer first and then provides the reasoning versus generating the reasoning process first and then the conclusion. Inspired by this, we propose a new benchmark method for assessing LLM consistency: comparing responses generated through these two different approaches. This benchmark effectively identifies instances where LLMs fabricate answers and subsequently generate justifications. Furthermore, we introduce a novel and straightforward prompt strategy designed to mitigate this issue. Experimental results demonstrate that this strategy improves performance across various LLMs compared to direct questioning. This work not only sheds light on a critical flaw in LLMs but also offers a practical solution to enhance their reliability.
How tall do you think these men are? Women now using AI to catch men lying about being 6ft tall on dating apps - and here's how you can try it
ChatGPT has already been used to write essays, tell jokes and even write best man speeches. But it seems the helpful AI bot can even make sure people on dating apps aren't lying to you about their height. Women are screenshotting photos from dating app profiles, inserting them into ChatGPT and asking it to provide an estimate of how tall they are. Justine Moore, a venture capitalist in San Francisco, said the AI's estimates are accurate to within an inch โ not just for men but for women too. So it may be a good tool to size up your romantic interest before you arrange to meet.
Considerations for differentially private learning with large-scale public pretraining โ interview with Gautam Kamath
Florian Tramer, Gautam Kamath and Nicholas Carlini won an International Conference on Machine Learning (ICML2024) best paper award for their work Position: Considerations for Differentially Private Learning with Large-Scale Public Pretraining. Differential privacy is a rigorous and provable notion of data privacy. Among other things, training a machine learning model with differential privacy can prevent it from spitting out its training data. The issue is that training a model with differential privacy generally comes at a significant hit to a model's utility. By incorporating "public data" (i.e., data that is not subject to privacy constraints) into the training procedure, it can help alleviate this concern and increase the resulting model's utility.
Relational Graph Convolutional Networks Do Not Learn Sound Rules
Morris, Matthew, Cucala, David J. Tena, Grau, Bernardo Cuenca, Horrocks, Ian
Graph neural networks (GNNs) are frequently used to predict missing facts in knowledge graphs (KGs). Motivated by the lack of explainability for the outputs of these models, recent work has aimed to explain their predictions using Datalog, a widely used logic-based formalism. However, such work has been restricted to certain subclasses of GNNs. In this paper, we consider one of the most popular GNN architectures for KGs, R-GCN, and we provide two methods to extract rules that explain its predictions and are sound, in the sense that each fact derived by the rules is also predicted by the GNN, for any input dataset. Furthermore, we provide a method that can verify that certain classes of Datalog rules are not sound for the R-GCN. In our experiments, we train R-GCNs on KG completion benchmarks, and we are able to verify that no Datalog rule is sound for these models, even though the models often obtain high to near-perfect accuracy. This raises some concerns about the ability of R-GCN models to generalise and about the explainability of their predictions. We further provide two variations to the training paradigm of R-GCN that encourage it to learn sound rules and find a trade-off between model accuracy and the number of learned sound rules.
ELLA: Empowering LLMs for Interpretable, Accurate and Informative Legal Advice
Hu, Yutong, Luo, Kangcheng, Feng, Yansong
Despite remarkable performance in legal consultation exhibited by legal Large Language Models(LLMs) combined with legal article retrieval components, there are still cases when the advice given is incorrect or baseless. To alleviate these problems, we propose {\bf ELLA}, a tool for {\bf E}mpowering {\bf L}LMs for interpretable, accurate, and informative {\bf L}egal {\bf A}dvice. ELLA visually presents the correlation between legal articles and LLM's response by calculating their similarities, providing users with an intuitive legal basis for the responses. Besides, based on the users' queries, ELLA retrieves relevant legal articles and displays them to users. Users can interactively select legal articles for LLM to generate more accurate responses. ELLA also retrieves relevant legal cases for user reference. Our user study shows that presenting the legal basis for the response helps users understand better. The accuracy of LLM's responses also improves when users intervene in selecting legal articles for LLM. Providing relevant legal cases also aids individuals in obtaining comprehensive information.
BERT's Conceptual Cartography: Mapping the Landscapes of Meaning
Conceptual Engineers want to make words better. However, they often underestimate how varied our usage of words is. In this paper, we take the first steps in exploring the contextual nuances of words by creating conceptual landscapes -- 2D surfaces representing the pragmatic usage of words -- that conceptual engineers can use to inform their projects. We use the spoken component of the British National Corpus and BERT to create contextualised word embeddings, and use Gaussian Mixture Models, a selection of metrics, and qualitative analysis to visualise and numerically represent lexical landscapes. Such an approach has not yet been used in the conceptual engineering literature and provides a detailed examination of how different words manifest in various contexts that is potentially useful to conceptual engineering projects. Our findings highlight the inherent complexity of conceptual engineering, revealing that each word exhibits a unique and intricate landscape. Conceptual Engineers cannot, therefore, use a one-size-fits-all approach when improving words -- a task that may be practically intractable at scale.
Congratulations to the #IJCAI2024 distinguished paper award winners
The International Joint Conference on Artificial Intelligence (IJCAI) distinguished paper awards recognise some of the best papers presented at the conference each year. This year, during the conference opening ceremony, three articles were named as distinguished papers. Abstract: As digital marketplaces and services continue to expand, it is crucial to maintain a safe and fair environment for all users. This requires implementing fairness constraints into the sequential decision-making processes of these platforms to ensure equal treatment. However, this can be challenging as these processes often need to solve NP-complete problems with exponentially large decision spaces at each time step.