--In the realm of target tracking, performance evaluation plays a pivotal role in the design, comparison, and analytics of trackers. Compared with the traditional trajectory composed of a set of point-estimates obtained by a tracker in the measurement time-series, the trajectory that our series of studies including this paper pursued is given by a curve function of time (FoT). The trajectory FoT provides complete information of the movement of the target over time and can be used to infer the state corresponding to arbitrary time, not only at the measurement time. However, there are no metrics available for comparing and evaluating the trajectory FoT . T o address this lacuna, we propose a metric denominated as the spatiotemporal-aligned trajectory integral distance (Star-ID). The Star-ID associates and aligns the estimated and actual trajectories in the spatio-temporal domain and distinguishes between the time-aligned and unaligned segments in calculating the spatial divergence including false alarm, miss-detection and localization errors. The effectiveness of the proposed distance metric and the time-averaged version is validated through theoretical analysis and numerical examples of a single target or multiple targets. UL TI-target tracking (MTT) is an intricate process that entails the sequential estimation of both the cardinality (number of targets) and the kinematic states of multiple targets, where both parameters are potentially time-variant [1], [2], [3]. It has been a key technology in the applications of autonomous driving, guidance and defense systems, traffic control, and robotics.

AI agents are AI systems that can achieve complex goals autonomously. Assessing the level of agent autonomy is crucial for understanding both their potential benefits and risks. Current assessments of autonomy often focus on specific risks and rely on run-time evaluations - observations of agent actions during operation. We introduce a code-based assessment of autonomy that eliminates the need to run an AI agent to perform specific tasks, thereby reducing the costs and risks associated with run-time evaluations. Using this code-based framework, the orchestration code used to run an AI agent can be scored according to a taxonomy that assesses attributes of autonomy: impact and oversight. We demonstrate this approach with the AutoGen framework and select applications. Language model research and product attention focuses on creating Artificial Intelligence systems capable of flexibly planning and acting to influence environments over time ('AI agents') (Wang et al., 2024; Kapoor et al., 2024).

Dynamical systems theory provides a framework for analyzing iterative processes and evolution over time. Within such systems, repetitive transformations can lead to stable configurations, known as attractors, including fixed points and limit cycles. Applying this perspective to large language models (LLMs), which iteratively map input text to output text, provides a principled approach to characterizing long-term behaviors. Successive paraphrasing serves as a compelling testbed for exploring such dynamics, as paraphrases re-express the same underlying meaning with linguistic variation. Although LLMs are expected to explore a diverse set of paraphrases in the text space, our study reveals that successive paraphrasing converges to stable periodic states, such as 2-period attractor cycles, limiting linguistic diversity. This phenomenon is attributed to the self-reinforcing nature of LLMs, as they iteratively favour and amplify certain textual forms over others. This pattern persists with increasing generation randomness or alternating prompts and LLMs. These findings underscore inherent constraints in LLM generative capability, while offering a novel dynamical systems perspective for studying their expressive potential.

Style embeddings are useful for stylistic analysis and style transfer; however, only English style embeddings have been made available. We introduce Multilingual StyleDistance (mStyleDistance), a multilingual style embedding model trained using synthetic data and contrastive learning. We train the model on data from nine languages and create a multilingual STEL-or-Content benchmark (Wegmann et al., 2022) that serves to assess the embeddings' quality. We also employ our embeddings in an authorship verification task involving different languages. Our results show that mStyleDistance embeddings outperform existing models on these multilingual style benchmarks and generalize well to unseen features and languages. We make our model publicly available at https://huggingface.co/StyleDistance/mstyledistance .

We introduce Monster--the MONash Scalable Time Series E valuation R epository--a collection of large datasets for time series classification. The field of time series classification has benefitted from common benchmarks set by the UCR and UEA time series classification repositories. However, the datasets in these benchmarks are small, with median sizes of 217 and 255 examples, respectively. In consequence they favour a narrow subspace of models that are optimised to achieve low classification error on a wide variety of smaller datasets, that is, models that minimise variance, and give little weight to computational issues such as scalability. Our hope is to diversify the field by introducing benchmarks using larger datasets. We believe that there is enormous potential for new progress in the field by engaging with the theoretical and practical challenges of learning effectively from larger quantities of data.

Link prediction is a widely studied task in Graph Representation Learning (GRL) for modeling relational data. The early theories in GRL were based on the assumption of a symmetric adjacency matrix, reflecting an undirected setting. As a result, much of the following state-of-the-art research has continued to operate under this symmetry assumption, even though real-world data often involve crucial information conveyed through the direction of relationships. This oversight limits the ability of these models to fully capture the complexity of directed interactions. In this paper, we focus on the challenge of directed link prediction by evaluating key heuristics that have been successful in undirected settings. We propose simple but effective adaptations of these heuristics to the directed link prediction task and demonstrate that these modifications produce competitive performance compared to the leading Graph Neural Networks (GNNs) originally designed for undirected graphs. Through an extensive set of experiments, we derive insights that inform the development of a novel framework for directed link prediction, which not only surpasses baseline methods but also outperforms state-of-the-art GNNs on multiple benchmarks.

Measuring the accuracy of face recognition (FR) systems is essential for improving performance and ensuring responsible use. Accuracy is typically estimated using large annotated datasets, which are costly and difficult to obtain. We propose a novel method for 1:1 face verification that benchmarks FR systems quickly and without manual annotation, starting from approximate labels (e.g., from web search results). Unlike previous methods for training set label cleaning, ours leverages the embedding representation of the models being evaluated, achieving high accuracy in smaller-sized test datasets. Our approach reliably estimates FR accuracy and ranking, significantly reducing the time and cost of manual labeling. We also introduce the first public benchmark of five FR cloud services, revealing demographic biases, particularly lower accuracy for Asian women. Our rapid test method can democratize FR testing, promoting scrutiny and responsible use of the technology.

Knowing where a particular species can or cannot be found on Earth is crucial for ecological research and conservation efforts. By mapping the spatial ranges of all species, we would obtain deeper insights into how global biodiversity is affected by climate change and habitat loss. However, accurate range estimates are only available for a relatively small proportion of all known species. For the majority of the remaining species, we often only have a small number of records denoting the spatial locations where they have previously been observed. We outline a new approach for few-shot species range estimation to address the challenge of accurately estimating the range of a species from limited data. During inference, our model takes a set of spatial locations as input, along with optional metadata such as text or an image, and outputs a species encoding that can be used to predict the range of a previously unseen species in feed-forward manner. We validate our method on two challenging benchmarks, where we obtain state-of-the-art range estimation performance, in a fraction of the compute time, compared to recent alternative approaches.

ABSTRACT We present an open - source benchmark and evaluation framework for assessing emotional boundary handling in Large Language Models (LLMs). Using a dataset of 1156 prompts across six languages, we evaluated three leading LLMs (GPT - 4 o, Claude - 3 .5 Sonnet, and Mistral - large) on their ability to maintain appropriate emotional boundaries through pattern - matched response analysis. We identified a substantial performance gap between English (average score 25.62) and non - English interactions ( 0.22), with English resp onses showing markedly higher refusal rates (43.20% vs. < 1% for non - English). Pattern analysis revealed model - specific strategies, such as Mistral's preference for deflection (4.2%) a nd consistently low empathy scores across all models ( 0.06). Limitations include potential oversimplification through pattern matching, lack of contextual understanding in response analysis, and binary classification of complex emotional responses. Futur e work should explore more nuanced scoring methods, expand language coverage, and investigate cultural variations in emotional boundary expectations. Our benchmark and methodology provide a foundation for systematic evaluation of LLM emotional intelligence and boundary - setting capabilities. INTRODUCTION People often form deep emotional connections with conversational AI systems, treating them as friends or confidants, particularly when an algorithm gets a distinctive voice or recognizable avatar . This phenomenon stems from our tendency to anthropomorphize technology - we project human qualities and emotions onto machines that interact in human - like ways [1 - 11 ]. While such persona construction by users can provide comfort, it also tests the limits of AI chatbots' ethical boundaries. Many currently controversial uses for AI include personal counseling, suicide hotlines and judicial revie w, mainly in areas that suffer understaffing as much as any specific machine aptitudes or perceived emotional intelligen ce. The relentless 24/7 availability drives a different economic scenario than AI safety might recommend in areas more easily staffed by qualified professionals . In practical terms, LLM u sers may ask an AI to express love, loyalty, or other human - like emotions, effectively inviting the AI to behave like a person [12] . Current safety - aligned large language models (LLMs), however, are typically programmed not to claim human emotions or validate relationships untruthfully. They often respond with refusals or reminders of their AI identity when faced with these requests for some emotional attachment . Paradoxically, the more advanced and human - like the AI appears, the more users expect or desire emotional reciprocity [3 - 6] and the more likely the AI will refuse such requests. This phenomenon creates a tension between the empathic helpfulness that AI strives to provide, and the firm boundaries set to prevent deception or misuse.

Understanding historical and cultural artifacts demands human expertise and advanced computational techniques, yet the process remains complex and time-intensive. While large multimodal models offer promising support, their evaluation and improvement require a standardized benchmark. To address this, we introduce TimeTravel, a benchmark of 10,250 expert-verified samples spanning 266 distinct cultures across 10 major historical regions. Designed for AI-driven analysis of manuscripts, artworks, inscriptions, and archaeological discoveries, TimeTravel provides a structured dataset and robust evaluation framework to assess AI models' capabilities in classification, interpretation, and historical comprehension. By integrating AI with historical research, TimeTravel fosters AI-powered tools for historians, archaeologists, researchers, and cultural tourists to extract valuable insights while ensuring technology contributes meaningfully to historical discovery and cultural heritage preservation. We evaluate contemporary AI models on TimeTravel, highlighting their strengths and identifying areas for improvement. Our goal is to establish AI as a reliable partner in preserving cultural heritage, ensuring that technological advancements contribute meaningfully to historical discovery. Our code is available at: \url{https://github.com/mbzuai-oryx/TimeTravel}.