3acb2a202ae4bea8840224e6fce16fd0-AuthorFeedback.pdf

Twootherfactorsexplainthisgap: 1)BERTcontextual10 embeddings are known to perform surprisingly poorly on tasks beyond the word-level (see e.g. R2: Prism layer transforms are fixed; how does this compare to a learned transform? R3: "The way of dividing the embeddings into 5 sectors seems a bit naive" We made this choice primarily to27 enable clear comparisons between tasks atdifferent linguistic scales. Reviewers also offered a number of other suggestions which we are grateful for and will incorporate into the final48 versionofourpaper.49