Goto

Collaborating Authors

 daily activity


Large Language Models as Urban Residents: An LLM Agent Framework for Personal Mobility Generation

Neural Information Processing Systems

This paper introduces a novel approach using Large Language Models (LLMs) integrated into an agent framework for flexible and effective personal mobility generation. LLMs overcome the limitations of previous models by effectively processing semantic data and offering versatility in modeling various tasks.


LearningState-AwareVisualRepresentationsfrom AudibleInteractions

Neural Information Processing Systems

We propose a self-supervised algorithm to learn representations from egocentric video data. Recently, significant efforts have been made to capture humans interacting with their own environments as they go about their daily activities. In result, several large egocentric datasets of interaction-rich multi-modal data have emerged. However, learning representations from videos can be challenging. First, given the uncurated nature of long-form continuous videos, learning effectiverepresentations require focusing onmoments intimewhen interactions take place. Second, visual representations of daily activities should be sensitive to changes in the state of the environment. However, current successful multimodal learning frameworks encourage representation invariance over time.


Learning State-Aware Visual Representations from Audible Interactions

Neural Information Processing Systems

We propose a self-supervised algorithm to learn representations from egocentric video data. Recently, significant efforts have been made to capture humans interacting with their own environments as they go about their daily activities. In result, several large egocentric datasets of interaction-rich multi-modal data have emerged. However, learning representations from videos can be challenging. First, given the uncurated nature of long-form continuous videos, learning effective representations require focusing on moments in time when interactions take place.


Neurotremor: A wearable Supportive Device for Supporting Upper Limb Muscle Function

arXiv.org Artificial Intelligence

A sensor-fused wearable assistance prototype for upper-limb function (triceps brachii and extensor pollicis brevis) is presented. The device integrates surface electromyography (sEMG), an inertial measurement unit (IMU), and flex/force sensors on an M5StickC plus an ESP32-S3 compute hub. Signals are band-pass and notch filtered; features (RMS, MAV, zero-crossings, and 4-12 Hz tremor-band power) are computed in 250 ms windows and fed to an INT8 TensorFlow Lite Micro model. Control commands are bounded by a control-barrier-function safety envelope and delivered within game-based tasks with lightweight personalization. In a pilot technical feasibility evaluation with healthy volunteers (n = 12) performing three ADL-oriented tasks, tremor prominence decreased (Delta TI = -0.092, 95% CI [-0.102, -0.079]), range of motion increased (+12.65%, 95% CI [+8.43, +13.89]), repetitions rose (+2.99 min^-1, 95% CI [+2.61, +3.35]), and the EMG median-frequency slope became less negative (Delta = +0.100 Hz/min, 95% CI [+0.083, +0.127]). The sensing-to-assist loop ran at 100 Hz with 8.7 ms median on-device latency, 100% session completion, and 0 device-related adverse events. These results demonstrate technical feasibility of embedded, sensor-fused assistance for upper-limb function; formal patient studies under IRB oversight are planned.


Large Language Models as Urban Residents: An LLM Agent Framework for Personal Mobility Generation Jiawei Wang

Neural Information Processing Systems

This paper introduces a novel approach using Large Language Models (LLMs) integrated into an agent framework for flexible and effective personal mobility generation. LLMs overcome the limitations of previous models by effectively processing semantic data and offering versatility in modeling various tasks.


Enhancing Smart Environments with Context-Aware Chatbots using Large Language Models

arXiv.org Artificial Intelligence

This work presents a novel architecture for context-aware interactions within smart environments, leveraging Large Language Models (LLMs) to enhance user experiences. Our system integrates user location data obtained through UWB tags and sensor-equipped smart homes with real-time human activity recognition (HAR) to provide a comprehensive understanding of user context. This contextual information is then fed to an LLM-powered chatbot, enabling it to generate personalised interactions and recommendations based on the user's current activity and environment. This approach moves beyond traditional static chatbot interactions by dynamically adapting to the user's real-time situation. A case study conducted from a real-world dataset demonstrates the feasibility and effectiveness of our proposed architecture, showcasing its potential to create more intuitive and helpful interactions within smart homes. The results highlight the significant benefits of integrating LLM with real-time activity and location data to deliver personalised and contextually relevant user experiences.


Investigating an Intelligent System to Monitor \& Explain Abnormal Activity Patterns of Older Adults

arXiv.org Artificial Intelligence

Despite the growing potential of older adult care technologies, the adoption of these technologies remains challenging. In this work, we conducted a focus-group session with family caregivers to scope designs of the older adult care technology. We then developed a high-fidelity prototype and conducted its qualitative study with professional caregivers and older adults to understand their perspectives on the system functionalities. This system monitors abnormal activity patterns of older adults using wireless motion sensors and machine learning models and supports interactive dialogue responses to explain abnormal activity patterns of older adults to caregivers and allow older adults proactively sharing their status with caregivers for an adequate intervention. Both older adults and professional caregivers appreciated that our system can provide a faster, personalized service while proactively controlling what information is to be shared through interactive dialogue responses. We further discuss other considerations to realize older adult technology in practice.


Learning State-Aware Visual Representations from Audible Interactions

Neural Information Processing Systems

We propose a self-supervised algorithm to learn representations from egocentric video data. Recently, significant efforts have been made to capture humans interacting with their own environments as they go about their daily activities. In result, several large egocentric datasets of interaction-rich multi-modal data have emerged. However, learning representations from videos can be challenging. First, given the uncurated nature of long-form continuous videos, learning effective representations require focusing on moments in time when interactions take place.


Benchmarking LLMs and SLMs for patient reported outcomes

arXiv.org Artificial Intelligence

LLMs have transformed the execution of numerous tasks, including those in the medical domain. Among these, summarizing patient-reported outcomes (PROs) into concise natural language reports is of particular interest to clinicians, as it enables them to focus on critical patient concerns and spend more time in meaningful discussions. While existing work with LLMs like GPT-4 has shown impressive results, real breakthroughs could arise from leveraging SLMs as they offer the advantage of being deployable locally, ensuring patient data privacy and compliance with healthcare regulations. This study benchmarks several SLMs against LLMs for summarizing patient-reported Q\&A forms in the context of radiotherapy. Using various metrics, we evaluate their precision and reliability. The findings highlight both the promise and limitations of SLMs for high-stakes medical tasks, fostering more efficient and privacy-preserving AI-driven healthcare solutions.


Simulating Human-like Daily Activities with Desire-driven Autonomy

arXiv.org Artificial Intelligence

Existing task-oriented AI agents often depend on explicit instructions or external rewards, limiting their ability to be driven by intrinsic motivations like humans. In this paper, we present a desire-driven autonomy framework to guide a Large Language Model-based (LLM-based) agent to simulate human-like daily activities. In contrast to previous agents, our Desire-driven Autonomous Agent (D2A) operates on the principle of intrinsic desire, allowing it to propose and select tasks that fulfill its motivational framework autonomously. Inspired by the Theory of Needs, the motivational framework incorporates an understanding of human-like desires, such as the need for social interaction, personal fulfillment, and self-care. Utilizing a desire-driven task generation mechanism, the agent evaluates its current state and takes a sequence of activities aligned with its intrinsic motivations. Through simulations, we demonstrate that our Desire-driven Autonomous Agent (D2A) generates coherent, contextually relevant daily activities while exhibiting variability and adaptability similar to human behavior. A comparative analysis with other LLM-based frameworks demonstrates that our approach significantly enhances the rationality of the simulated activities.