Large Language Model
Causal Deciphering and Inpainting in Spatio-Temporal Dynamics via Diffusion Model
Spatio-temporal (ST) prediction has garnered a De facto attention in earth sciences, such as meteorological prediction, human mobility perception. However, the scarcity of data coupled with the high expenses involved in sensor deployment results in notable data imbalances.
UniMTS: Unified Pre-training for Motion Time Series
Typically, existing models are trained and tested on the same dataset, leading to poor generalizability across variations in device location, device mounting orientation, and human activity type.
SlimGPT: Layer-wise Structured Pruning for Large Language Models Gui Ling, Ziyang Wang, Yuliang Y an
Structured pruning is an effective method to balance model performance with efficiency, but performance restoration under computational resource constraints is a principal challenge in pruning LLMs. Therefore, we present a low-cost and fast structured pruning method for LLMs named SlimGPT based on the Optimal Brain Surgeon framework.
StreamBench: Towards Benchmarking Continuous Improvement of Language Agents
To address this gap, we introduce StreamBench, a pioneering benchmark designed to evaluate the continuous improvement of LLM agents over an input-feedback sequence.
Risk-Averse Fine-tuning of Large Language Models Sapana Chaudhary Amazon Web Services (A WS)
The deployment of large language models (LLMs) is witnessing remarkable growth across both personal and professional domains [Nakano et al., 2021, Touvron et al., 2023].
Transformers are Minimax Optimal Nonparametric In-Context Learners Juno Kim 1,2 T ai Nakamaki 1 T aiji Suzuki 1,2 1 University of Tokyo 2
Large language models (LLMs) have demonstrated remarkable capabilities in understanding and generating natural language data.
Advancing Tool-Augmented Large Language Models: Integrating Insights from Errors in Inference Trees Sijia Chen 1, 2, Yibo Wang 1, 2, Yi-Feng Wu3 Qing-Guo Chen
Tool-augmented large language models (LLMs) leverage tools, often in the form of APIs, to improve their reasoning capabilities on complex tasks. This enables them to act as intelligent agents interacting with the real world. The recently introduced ToolLLaMA model by Qin et al. [ 2023 ] utilizes the depth-first search-based decision tree (DFSDT) mechanism for multi-step reasoning with 16000+ real-world APIs, effectively enhancing the performance of tool-augmented LLMs compared to traditional chain reasoning mechanisms. However, their approach only employs successful paths from decision trees (also called inference trees) for supervised fine-tuning (SFT), missing out on the potential learning opportunities from failed paths. Inspired by this, we propose an inference trajectory optimization framework based on preference learning to address this limitation.
BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack
To bridge this gap, we introduce the BABILong benchmark, designed to test language models' ability to reason across facts distributed in extremely
UniAR: A Unified model for predicting human Attention and Responses on visual content
Progress in human behavior modeling involves understanding both implicit, early-stage perceptual behavior, such as human attention, and explicit, later-stage behavior, such as subjective preferences or likes.