Wefurther propose attention sharing strategies to facilitate parameter sharing and specialization in multilingual and multi-domain sequencemodeling.

The Transformer model has achieved state-of-the-art performance in many sequence modeling tasks. However,howtoleverage model capacity with largeor variable depths is still an open challenge.

We thank the reviewers for their feedback. We will reflect reviewer's comments and our response in the revision. Reviewers showed concern on the novelty and the accuracy. DA is more effective when the task is more challenging. On the other hand, we find DA effective as well when the amount of labeled data is small.