Assessing Non-autoregressive Alignment in Neural Machine Translation via Word Reordering

Chun Hin Tse, Ester S.M. Leung, William K. Cheung

Research output: Chapter in book/report/conference proceedingConference proceedingpeer-review


Recent work on non-autoregressive neural machine translation (NAT) that leverages alignment information to explicitly reduce the modality of target distribution has reported comparable performance with counterparts that tackle multi-modality problem by implicitly modeling dependencies. Effectiveness in handling alignment is vital for models that follow this approach, where a token reordering mechanism is typically involved and plays a vital role. We review the reordering capability of the respective mechanisms in recent NAT models, and our experimental results show that their performance is sub-optimal. We propose to learn a non-autoregressive language model (NALM) based on transformer which can be combined with Viterbi decoding to achieve better reordering performance. We evaluate the proposed NALM using the PTB dataset where sentences with words permuted in different ways are expected to have their ordering recovered. Our empirical results show that the proposed method can outperform the state-of-the-art reordering mechanisms under different word permutation settings, with a 2-27 BLEU improvement, suggesting high potential for word alignment in NAT.

Original languageEnglish
Title of host publicationFindings of the Association for Computational Linguistics: EMNLP 2022
EditorsYoav Goldberg, Zornitsa Kozareva, Yue Zhang
PublisherAssociation for Computational Linguistics (ACL)
Number of pages7
Publication statusPublished - Dec 2022
EventThe 2022 Conference on Empirical Methods in Natural Language Processing - Abu Dhabi, United Arab Emirates
Duration: 7 Dec 202211 Dec 2022


ConferenceThe 2022 Conference on Empirical Methods in Natural Language Processing
Abbreviated titleEMNLP 2022
Country/TerritoryUnited Arab Emirates
CityAbu Dhabi
Internet address

Scopus Subject Areas

  • Computational Theory and Mathematics
  • Computer Science Applications
  • Information Systems

User-Defined Keywords

  • Neural machine translation
  • Non-autoregressive alignment
  • Word reordering


Dive into the research topics of 'Assessing Non-autoregressive Alignment in Neural Machine Translation via Word Reordering'. Together they form a unique fingerprint.

Cite this