Deep Learning
Unchosen Experts Can Contribute Too: Unleashing MoE Models ' Power by Self-Contrast
Mixture-of-Experts (MoE) has emerged as a prominent architecture for scaling model size while maintaining computational efficiency.
Aligning LLM Agents by Learning Latent Preference from User Edits Ge Gao Alexey T aymanov Eduardo Salinas Paul Mineiro Dipendra Misra Department of Computer Science, Cornell University
The inferred user preference descriptions are used to define prompts for generating responses in the future.
dattri: A Library for Efficient Data Attribution Junwei Deng 1* Ting-Wei Li
Data attribution methods aim to quantify the influence of individual training samples on the prediction of artificial intelligence (AI) models.
Sharpness-diversity tradeoff: improving flat ensembles with SharpBalance
Building on this, our study investigates the interplay between sharpness and diversity within deep ensembles, illustrating their crucial role in robust generalization to both in-distribution (ID) and out-of-distribution (OOD) data.
SELECT: A Large-Scale Benchmark of Data Curation Strategies for Image Classification
Our findings show interesting trends, particularly pertaining to recent methods for data curation such as synthetic data generation and lookup based on CLIP embeddings. We show that although these strategies are highly competitive for certain tasks, the curation strategy used to assemble the original ImageNet-1K dataset remains the gold standard. We anticipate that our benchmark can illuminate the path for new methods to further reduce the gap.