Abstract
While textual reviews have become prominent in many recommendation-based systems, automated frameworks to provide relevant visual cues against text reviews where pictures are not available is a new form of task confronted by data mining and machine learning researchers. Suggestions of pictures that are relevant to the content of a review could significantly benefit the users by increasing the effectiveness of a review. We propose a deep learning-based framework to automatically: (1) tag the images available in a review dataset, (2) generate a caption for each image that does not have one, and (3) enhance each review by recommending relevant images that might not be uploaded by the corresponding reviewer. We evaluate the proposed framework using the Yelp Challenge Dataset. While a subset of the images in this particular dataset are correctly captioned, the majority of the pictures do not have any associated text. Moreover, there is no mapping between reviews and images. Each image has a corresponding business-tag where the picture was taken, though. The overall data setting and unavailability of crucial pieces required for a mapping make the problem of recommending images for reviews a major challenge. Qualitative and quantitative evaluations indicate that our proposed framework provides high quality enhancements through automatic captioning, tagging, and recommendation for mapping reviews and images.
Abstract (translated by Google)
尽管在许多基于推荐的系统中文本评论已经变得突出，但是提供相关的视觉提示以对照不存在图片的文本评论的自动化框架是数据挖掘和机器学习研究者面临的新形式的任务。对与评论内容相关的图片的建议可以通过提高评论的有效性而显着地使用户受益。我们提出了一个深入的基于学习的框架来自动地：（1）在评论数据集中标记可用图像，（2）为每个没有图像的图像生成标题，（3）通过推荐相关图像可能不会被相应的评论者上传。我们使用Yelp挑战数据集来评估建议的框架。虽然在此特定数据集中的图像子集正确标题，大多数图片没有任何关联的文字。而且，评论和图像之间没有映射。尽管如此，每张图片都有相应的商业标签。整体数据设置和映射所需的关键部分的无法使用使得推荐用于评论的图像成为主要挑战。定性和定量评估表明，我们提出的框架通过自动字幕，标记和映射评论和图像推荐提供高质量的增强。
URL
https://arxiv.org/abs/1606.07496