Machine Translation
Novel positional encodings to enable tree-based transformers
Motivated by this property, we propose a method to extend transformers to tree-structured data, enabling sequence-totree, tree-to-sequence, and tree-to-tree mappings. Our approach abstracts the transformer'ssinusoidal positional encodings, allowing ustoinstead useanovel positional encoding scheme to represent node positions within trees.
Mixtape: Breaking the Softmax Bottleneck Efficiently
Zhilin Yang, Thang Luong, Russ R. Salakhutdinov, Quoc V. Le
Neural Information Processing Systems http://nips.cc/
Can Unconditional Language Models Recover Arbitrary Sentences?
Nishant Subramani, Samuel Bowman, Kyunghyun Cho
Neural Information Processing Systems http://nips.cc/
Universal Approximation of Input-Output Maps by Temporal Convolutional Nets
Neural Information Processing Systems http://nips.cc/
GraphFormers
Tolearnhigh-quality representation for textual graph, techniques on natural language understanding and graph representation need to be jointly leveraged.