Deep Learning
Approximation Rate of the Transformer Architecture for Sequence Modeling
In this work, we investigate the approximation rate results for the Transformer architectures on general sequence to sequence target relationships.
Changing the Training Data Distribution to Reduce Simplicity Bias Improves In-distribution Generalization
By studying a two-layer CNN, we rigorously prove that SAM learns different features more uniformly, particularly in early epochs.
LLM Evaluators Recognize and Favor Their Own Generations
But new biases are introduced due to the same LLM acting as both the evaluator and the evaluatee.
Group and Shuffle: Efficient Structured Orthogonal Parametrization
In this paper, we introduce a new class of structured matrices, which unifies and generalizes structured classes from previous works.
Factorized Diffusion Architectures for Unsupervised Image Generation and Segmentation
This work was completed while Xin Y uan was a PhD student at the University of Chicago.
FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision Jay Shah
In this work, we build on the work of Dao et al.
Quantitatively Understanding the Bitter Lesson Through Capabilities Trajectories
In an extensive analysis of existing safety benchmarks, we find that variance in model performance on many safety benchmarks is largely explained by the capabilities component.
Revisiting Ensembling in One-Shot Federated Learning
The overall learning procedure is illustrated in Figure 1, alongside iterative and one-shot FL. 1.1 Our Contributions
GraphMorph: Tubular Structure Extraction by Morphing Predicted Graphs
GraphMorph comprises two main components: a Graph Decoder and a Morph Module.