Abstract
In this paper we consider the problem of continuously discovering image contents by actively asking image based questions and subsequently answering the questions being asked. The key components include a Visual Question Generation (VQG) module and a Visual Question Answering module, in which Recurrent Neural Networks (RNN) and Convolutional Neural Network (CNN) are used. Given a dataset that contains images, questions and their answers, both modules are trained at the same time, with the difference being VQG uses the images as input and the corresponding questions as output, while VQA uses images and questions as input and the corresponding answers as output. We evaluate the self talk process subjectively using Amazon Mechanical Turk, which show effectiveness of the proposed method.
Abstract (translated by Google)
在本文中，我们考虑通过主动提出基于图像的问题来不断发现图像内容的问题，并随后回答所提出的问题。关键部分包括视觉问题生成（VQG）模块和视觉问答模块，其中使用递归神经网络（RNN）和卷积神经网络（CNN）。给定一个包含图像，问题及其答案的数据集，两个模块同时被训练，不同之处在于VQG使用图像作为输入并将相应的问题作为输出，而VQA使用图像和问题作为输入以及相应的答案作为输出。我们使用Amazon Mechanical Turk主观地评估自我谈话过程，其显示了所提出的方法的有效性。
URL
https://arxiv.org/abs/1512.03460