Abstract
Recurrent neural network architectures combining with attention mechanism, or neural attention model, have shown promising performance recently for the tasks including speech recognition, image caption generation, visual question answering and machine translation. In this paper, neural attention model is applied on two sequence classification tasks, dialogue act detection and key term extraction. In the sequence labeling tasks, the model input is a sequence, and the output is the label of the input sequence. The major difficulty of sequence labeling is that when the input sequence is long, it can include many noisy or irrelevant part. If the information in the whole sequence is treated equally, the noisy or irrelevant part may degrade the classification performance. The attention mechanism is helpful for sequence classification task because it is capable of highlighting important part among the entire sequence for the classification task. The experimental results show that with the attention mechanism, discernible improvements were achieved in the sequence labeling task considered here. The roles of the attention mechanism in the tasks are further analyzed and visualized in this paper.
Abstract (translated by Google)
结合注意机制或神经注意模型的递归神经网络体系结构最近在包括语音识别，图像标题生成，视觉问题解答和机器翻译在内的任务中表现出有希望的性能。本文将神经注意模型应用于两个序列分类任务，对话行为检测和关键词提取。在序列标签任务中，模型输入是一个序列，输出是输入序列的标签。序列标记的主要难点在于，当输入序列较长时，可能包含很多噪声或不相关的部分。如果整个序列中的信息被同等对待，则噪声或不相关部分可能会降低分类性能。注意机制对于序列分类任务是有帮助的，因为它能够突出整个序列中的重要部分进行分类任务。实验结果表明，在注意机制下，在这里考虑的序列标注任务中取得了明显的改进。关注机制在任务中的作用进一步分析和可视化。
URL
https://arxiv.org/abs/1604.00077