Goto

Collaborating Authors

 contextual awareness


Line of Duty: Evaluating LLM Self-Knowledge via Consistency in Feasibility Boundaries

arXiv.org Artificial Intelligence

As LLMs grow more powerful, their most profound achievement may be recognising when to say "I don't know". Existing studies on LLM self-knowledge have been largely constrained by human-defined notions of feasibility, often neglecting the reasons behind unanswerability by LLMs and failing to study deficient types of self-knowledge. This study aims to obtain intrinsic insights into different types of LLM self-knowledge with a novel methodology: allowing them the flexibility to set their own feasibility boundaries and then analysing the consistency of these limits. We find that even frontier models like GPT-4o and Mistral Large are not sure of their own capabilities more than 80% of the time, highlighting a significant lack of trustworthiness in responses. Our analysis of confidence balance in LLMs indicates that models swing between overconfidence and conservatism in feasibility boundaries depending on task categories and that the most significant self-knowledge weaknesses lie in temporal awareness and contextual understanding. These difficulties in contextual comprehension additionally lead models to question their operational boundaries, resulting in considerable confusion within the self-knowledge of LLMs. We make our code and results available publicly at https://github.com/knowledge-verse-ai/LLM-Self_Knowledge_Eval


The Reliability of LLMs for Medical Diagnosis: An Examination of Consistency, Manipulation, and Contextual Awareness

arXiv.org Artificial Intelligence

Universal healthcare access is critically needed, especially in resource-limited settings. Large Language Models (LLMs) offer promise for democratizing healthcare with advanced diagnostics, but their reliability requires thorough evaluation, especially in trust-dependent environments. This study assesses LLMs' diagnostic reliability focusing on consistency, manipulation resilience, and contextual integration, crucial for safe and ethical use in universal healthcare. We evaluated leading LLMs using 52 patient cases, expanded into variants with demographic changes, symptom rewordings, and exam modifications, while keeping core diagnoses constant. Manipulation susceptibility was tested by inserting misleading narratives and irrelevant details. Contextual awareness was rvaluated by comparing diagnoses with and without patient history. We analyzed diagnostic change rates and response patterns across manipulations. LLMs showed perfect diagnostic consistency for identical data but significant manipulation susceptibility. Gemini had a 40% diagnosis change rate and ChatGPT 30% with irrelevant details. ChatGPT had a higher context influence rate (77.8% vs. Gemini's 55.6%), but both showed limited nuanced contextual integration, exhibiting anchoring bias by prioritizing salient data over context. LLMs' vulnerability to manipulation and limited contextual awareness pose challenges in clinical use. Unlike clinicians, they may overstate diagnostic certainty without validation. Safeguards and domain-specific designs are crucial for reliable healthcare applications. Broad clinical use without oversight is premature and risky. LLMs can enhance diagnostics with responsible use, but future research is needed to improve manipulation resistance and contextual understanding for safe healthcare democratization.


A Metasemantic-Metapragmatic Framework for Taxonomizing Multimodal Communicative Alignment

arXiv.org Artificial Intelligence

Drawing on contemporary pragmatist philosophy and linguistic theories on cognition, meaning, and communication, this paper presents a dynamic, metasemantic-metapragmatic taxonomy for grounding and conceptualizing human-like multimodal communicative alignment. The framework is rooted in contemporary developments of the three basic communicative capacities initially identified by American logician and pragmatist philosopher Charles Sanders Peirce: iconic (sensory and perceptual qualities), indexical (contextual and sociocultural associations), and rule-like (symbolic and intuitive reasoning). Expanding on these developments, I introduce the concept of indexical contextualization and propose the principle of "contextualization directionality" for characterizing the crucial metapragmatic capacity for maintaining, navigating, or transitioning between semantic and pragmatic modes of multimodal communication. I contend that current cognitive-social computational and engineering methodologies disproportionately emphasize the semantic/metasemantic domain, overlooking the pivotal role of metapragmatic indexicality in traversing the semantic-pragmatic spectrum of communication. The framework's broader implications for intentionality, identity, affect, and ethics in within-modal and cross-modal human-machine alignment are also discussed.


DynaCon: Dynamic Robot Planner with Contextual Awareness via LLMs

arXiv.org Artificial Intelligence

Mobile robots often rely on pre-existing maps for effective path planning and navigation. However, when these maps are unavailable, particularly in unfamiliar environments, a different approach become essential. This paper introduces DynaCon, a novel system designed to provide mobile robots with contextual awareness and dynamic adaptability during navigation, eliminating the reliance of traditional maps. DynaCon integrates real-time feedback with an object server, prompt engineering, and navigation modules. By harnessing the capabilities of Large Language Models (LLMs), DynaCon not only understands patterns within given numeric series but also excels at categorizing objects into matched spaces. This facilitates dynamic path planner imbued with contextual awareness. We validated the effectiveness of DynaCon through an experiment where a robot successfully navigated to its goal using reasoning. Source code and experiment videos for this work can be found at: https://sites.google.com/view/dynacon.


Know Your Conversational AI to its Barest Elements

#artificialintelligence

Conversational AI embellishes several innovative capabilities while being a programmatic and intelligent way of offering a conversational experience to mimic conversations with real people, through digital and telecommunication technologies, informed by rich datasets and intents, providing customers with informal, engaging experiences that mirror everyday language, digitally enabled products, platforms, and experiences relating to communication, sales and service consultations, as well as other customer services. Using conversational AI, organizations can provide personalized and differentiated experiences that build relationships with their customers. Each interaction can feel like a 1:1 conversation that is context-aware and informed by past interactions. Ever wondered, what inbred technologies drive such innovation. According to Deloitte's report, Conversational AI brings together eight technology components, including Natural Language Processing, Intent Recognition, Entity Recognition, Fulfilment, Voice Optimized Responses, Dynamic Text to Speech, Machine Learning, and Contextual Awareness.


The Spatial Web Is Coming -- Part 3 - AI Summary

#artificialintelligence

VERSES is Blockchain agnostic which means you can use multiple chains and even operate a hybrid data layer using both DLT technologies and the cloud. On September 21, 2022, VERSES introduced the creation of its new Artificial Intelligence Lab and Sensor Fusion Research Facility, showcasing their technology portfolio, including COSM, and providing an immersive space for data science and product development teams to cultivate advanced adaptive intelligence solutions required for translating diverse data into contextual awareness between humans, machines and AI in physical and digital spaces. Smart Contracts, at the heart of DLTs, are a programmable set of rules, stored on Blockchains, and run when certain pre-determined conditions are met. These automated, self-executing & immutable strings of code are recorded onto Distributed Ledger Blockchains, securing transactions & agreements by replacing static documents and the need for third party mediation. Through intelligent automation, Smart Contracts secure the management of property rights, spatial rights, proof of origination, verifiable traceability, and auto-execution of payments and transfer of assets, providing security, protecting privacy, and allowing risk-free interoperability -- all essential to a favorable and prosperous augmented and networked Web 3.0 experience.


Council Post: Neural Networks And Machine Learning Are Powering A New Era Of Perceptive Intelligence

#artificialintelligence

The human interface that connects us with machines -- the way we interact and control them -- has changed a lot over the years. From tactile methods like knobs, buttons, keyboards, pads and touch screens to more recent voice and visual command capabilities, we've adapted our devices to become more user-friendly and more humanlike by using more intuitive input techniques. We've all grown accustomed to the swipe, the pinch, the "Hey, Google," and the hand gesture to tell our devices what to do. But they still require the human element, a proactive direction by a person. A new generation -- indeed, ecosystem -- of devices, will be driven by interfaces that perceive your wants and needs.


Exploring the Abilities of Conversational AI and Its Technology Components

#artificialintelligence

Today, advances in automation, artificial intelligence (AI) and natural language processing (NLP) make it possible to design cost-efficient digital experiences. Now, where information can be purposeful, simple, and natural, customer conversations with organizations increasingly resemble conversations with employees in-person. According to Deloitte report, embellished with such innovative capabilities a programmatic and intelligent way of offering a conversational experience to mimic conversations with real people, through digital and telecommunication technologies, informed by rich data sets and intents, providing customers with informal, engaging experiences that mirror everyday language, digitally enabled products, platforms, and experiences relating to communication, sales and service consultations, as well as other customer services, is what we call Conversational AI. The Conversational AI market size is expected to grow from US$ 4.2 billion in 2019 to 15.7 billion by 2024, at a CAGR of 30.2%, during 2019-2024. Using conversational AI, organizations can provide personalized and differentiated experiences that build relationships with their customers. Each interaction can feel like a 1:1 conversation that is context-aware and informed by past interactions.


Is the Future of Smartphones a Walkie-Talkie That Talks Back?

Slate

Artificial intelligence is creeping into our smartphones in small, subtle ways. Google's Pixel 3, announced Tuesday, can answer robocalls on your behalf thanks to Google's Duplex technology and Google Assistant. Meanwhile, Android P, the latest operating system for Google's phones, can learn from how you interact with phone alerts to suggest stopping notifications for particular apps, reducing the amount of unnecessary intrusions your phone makes into your daily life. But there's another new phone in the pipeline that takes these kinds of developments further. By pairing them with more robust voice control, it may help fill in the picture of how we'll talk to the next generation of smartphones--and what they'll learn about us in order to talk back.


Mapbox Ushers In The Next Generation Of Mapping With New SDKs

Forbes - Tech

Right now, a lot of people are very excited about the future of technologies like AR, VR, AI, and autonomous vehicles. However, as I've written before, most of these technologies are relatively useless without contextual awareness. I have also written in the past about the importance of image sensors and how they enable AI and autonomous systems to better understand the world around them. Combining location awareness and vision is incredibly difficult and is fundamentally what enables app developers to anchor digital assets in the real world for augmented reality. There are currently only two companies capable of doing this-- Google and Mapbox.