Technology
Selective Generation for Controllable Language Models
Trustworthiness of generative language models (GLMs) is crucial in their deployment to critical decision making systems.
On the Dimensionality of Word Embedding
Neural Information Processing Systems http://nips.cc/
Social-BiGAT: Multimodal Trajectory Forecasting using Bicycle-GAN and Graph Attention Networks
Vineet Kosaraju, Amir Sadeghian, Roberto Martรญn-Martรญn, Ian Reid, Hamid Rezatofighi, Silvio Savarese
Neural Information Processing Systems http://nips.cc/
Efficient Graph Generation with Graph Recurrent Attention Networks
Renjie Liao, Yujia Li, Yang Song, Shenlong Wang, Will Hamilton, David K. Duvenaud, Raquel Urtasun, Richard Zemel
Neural Information Processing Systems http://nips.cc/
Nonparametric learning from Bayesian models with randomized objective functions
Simon Lyddon, Stephen Walker, Chris C. Holmes
This assumption is problematic, particularly in complex data environments.
HYDRA-FL: Hybrid Knowledge Distillation for Robust and Accurate Federated Learning
Data heterogeneity among Federated Learning (FL) users poses a significant challenge, resulting in reduced global model performance. The community has designed various techniques to tackle this issue, among which Knowledge Distillation (KD)-based techniques are common. While these techniques effectively improve performance under high heterogeneity, they inadvertently cause higher accuracy degradation under model poisoning attacks (known as attack amplification). This paper presents a case study to reveal this critical vulnerability in KD-based FL systems. We show why KD causes this issue through empirical evidence and use it as motivation to design a hybrid distillation technique. We introduce a novel algorithm, Hybrid Knowledge Distillation for Robust and Accurate FL (HYDRA-FL), which reduces the impact of attacks in attack scenarios by offloading some of the KD loss to a shallow layer via an auxiliary classifier.
Mental Sampling in Multimodal Representations
Jianqiao Zhu, Adam Sanborn, Nick Chater
Heated chains more readily traverse valleys in the probability landscape topropose movestofar-awaypeaks,whilethecolderchainsmakethe local steps that explore the current peak or patch.
Learningstepsizesforunfoldedsparsecoding
In this paper, we study the selection of adapted step sizes for ISTA. We showthat a simple step size strategy can improve the convergence rate of ISTA byleveraging the sparsity of the iterates. However, it is impractical in most largescale applications.