Deep Learning
Task Arithmetic in the Tangent Space: Improved Editing of Pre-Trained Models Guillermo Ortiz-Jimenez
We present a comprehensive study of task arithmetic in vision-language models and show that weight disentanglement is the crucial factor that makes it effective. This property arises during pre-training and manifests when distinct directions in weight space govern separate, localized regions in function space associated with the tasks.
ConStat: Performance-Based Contamination Detection in Large Language Models Jasper Dekoninck 1, Mark Niklas Mรผller 1,2, Martin Vechev 1 Department of Computer Science 1
Public benchmarks play an essential role in the evaluation of large language models.
A new perspective on building efficient and expressive 3D equivariant graph neural networks
Geometric deep learning enables the encoding of physical symmetries in modeling 3D objects.
Optimized Feature Generation for Tabular Data via LLMs with Decision Tree Reasoning
We use decision trees to convey this reasoning information, as they can be easily represented in natural language, effectively providing knowledge from prior experiments ( i.e., the impact of the generated features on performance) to
Unraveling the Gradient Descent Dynamics of Transformers
By analyzing the loss landscape of a single Transformer layer using Softmax and Gaussian attention kernels, our work provides concrete answers to these questions.
Differentiable sorting for censored time-to-event data
This bias becomes particularly noticeable when the study's endpoint can be inferred from the