Abstract
Neural machine translation has shown very promising results lately. Most NMT models follow the encoder-decoder framework. To make encoder-decoder models more flexible, attention mechanism was introduced to machine translation and also other tasks like speech recognition and image captioning. We observe that the quality of translation by attention-based encoder-decoder can be significantly damaged when the alignment is incorrect. We attribute these problems to the lack of distortion and fertility models. Aiming to resolve these problems, we propose new variations of attention-based encoder-decoder and compare them with other models on machine translation. Our proposed method achieved an improvement of 2 BLEU points over the original attention-based encoder-decoder.
Abstract (translated by Google)
神经机器翻译最近显示出非常有希望的结果。大多数NMT模型遵循编码器 - 解码器框架。为了使编解码器模型更加灵活，机器翻译引入了注意机制，还有其他的任务，如语音识别和图像字幕。我们观察到当对齐不正确时，基于注意力的编码器 - 解码器的翻译质量可能被显着损坏。我们将这些问题归因于缺乏扭曲和生育模式。针对这些问题，提出了基于注意的编解码器的新变体，并将其与其他机器翻译模型进行了比较。我们提出的方法比原来的基于注意力的编码器 - 解码器提高了2个BLEU点。
URL
https://arxiv.org/abs/1601.03317