Reviews: Self-attention with Functional Time Representation Learning

Originality: The application of self-attention in continuous-time event sequences is an interesting approach. The authors clearly note the shortcoming of self-attention when applied to such problems. They propose translation-invariant time kernel functions justified by classic function analysis theories and implement 4 new time embeddings that can be optimized by backpropagation and are compatible with self-attention. I believe the proposed time embeddings are novel and generalizable to other temporal tasks. Quality: Motivation from classic functional analysis theory [12] and [14]and developing differentiable time embeddings is the key contribution of this paper.