Abstract: The emergence of Transformer and its derivative models brings new opportunities to tasks of NLP (Natural Language Processing). Transformer is not only a separate model, but also the core of ...