Education
Efficient LLM Pretraining and Inference with Unlimited Context Length Xuezhe Ma ฯ Xiaomeng Y ang
The Transformer architecture (V aswani et al., 2017), despite its remarkable capabilities, faces challenges with quadratic
Trace is the Next AutoDiff: Generative Optimization with Rich Feedback, Execution Traces, and LLMs
We study a class of optimization problems motivated by automating the design and update of AI systems like coding assistants, robots, and copilots.
2DQuant: Low-bit Post-Training Quantization for Image Super-Resolution
In this work, we present a dual-stage low-bit post-training quantization (PTQ) method for image super-resolution, namely 2DQuant, which achieves efficient and accurate SR under low-bit quantization.
Improving Context-Aware Preference Modeling for Language Models
To address these challenges, we consider the two-step preference modeling procedure that first resolves the under-specification by selecting a context, and then evaluates preference with respect to the chosen context.
GEPS: Boosting Generalization in Parametric PDE Neural Solvers through Adaptive Conditioning
We demonstrate the versatility of our approach for both fully data-driven and for physics-aware neural solvers.
ViLCo-Bench: VIdeo Language COntinual learning Benchmark
This field is a relatively under-explored area, and establishing appropriate datasets is crucial for facilitating communication and research in this field.
WorldCoder,a Model-Based LLM Agent: Building World Models by Writing Code and Interacting with the Environment
We give a model-based agent that builds a Python program representing its knowledge of the world based on its interactions with the environment.
Continual Learning with Global Alignment
Specifically, we learn the data representation as a task-specific composition of pre-trained token representations shared across all tasks. Then the correlations between different tasks' data representations are grounded