Goto

Collaborating Authors

 dissimilarity







Compositional De-Attention Networks

Yi Tay, Anh Tuan Luu, Aston Zhang, Shuohang Wang, Siu Cheung Hui

Neural Information Processing Systems

Thispaperproposes a new quasi-attention that is compositional in nature, i.e., learning whether to add, subtract or nullify a certain vector when learning representations. This is strongly contrasted with vanilla attention, which simply re-weights input tokens.