Deep Learning
LoRA-GA: Low-Rank Adaptation with Gradient Approximation
Fine-tuning large-scale pretrained models is prohibitively expensive in terms of computational and memory costs. LoRA, as one of the most popular Parameter-Efficient Fine-Tuning (PEFT) methods, offers a cost-effective alternative by fine-tuning an auxiliary low-rank model that has significantly fewer parameters.
Instruction Tuning Large Language Models to Understand Electronic Health Records
This dataset covers various topics and is suitable for instruction-tuning general-purpose LLMs for diverse clinical use cases.
Instruction Tuning Large Language Models to Understand Electronic Health Records
This dataset covers various topics and is suitable for instruction-tuning general-purpose LLMs for diverse clinical use cases.
Taming the Long Tail in Human Mobility Prediction
This often results in data being sparse and biased towards popular locations, exhibiting a severe long-tail effect.
Controllable Heterogeneous Model Aggregation for Personalized Federated Learning
Several methods have emerged to aggregate diverse client models; however, they either lack the ability of personalization, raise privacy and security concerns, need prior knowledge, or ignore the capability and functionality of personalized models.
Dissecting Query-Key Interaction in Vision Transformers
Self-attention in vision transformers is often thought to perform perceptual grouping where tokens attend to other tokens with similar embeddings, which could correspond to semantically similar features of an object.
What Rotary Position Embedding Can T ell Us: Identifying Query and Key Weights Corresponding to Basic Syntactic or High-level Semantic Information
Transformer-based large language models (LLMs) have successfully handled various tasks.