BiT: RobustlyBinarizedMulti-distilledTransformer AnonymousAuthor(s) Affiliation Address email

Wekeep theteacher model fixed, while re-initializing thestudent model from9 the latest quantized version at each step. Here the P iWBi is summing up the values inWB, which can be pre-computed and stored as37 bias. QNLI Question Natural Language Inference (Wang et al., 2019) is a binary classification task50 whichisderivedfromtheStanfordQuestionAnsweringDataset(Rajpurkaretal.,2016). Semeval-2017task81 1: Semantic textual similarity-multilingual and cross-lingual focused evaluation.arXiv