Abstract
For some images, descriptions written by multiple people are consistent with each other. But for other images, descriptions across people vary considerably. In other words, some images are specific $-$ they elicit consistent descriptions from different people $-$ while other images are ambiguous. Applications involving images and text can benefit from an understanding of which images are specific and which ones are ambiguous. For instance, consider text-based image retrieval. If a query description is moderately similar to the caption (or reference description) of an ambiguous image, that query may be considered a decent match to the image. But if the image is very specific, a moderate similarity between the query and the reference description may not be sufficient to retrieve the image. In this paper, we introduce the notion of image specificity. We present two mechanisms to measure specificity given multiple descriptions of an image: an automated measure and a measure that relies on human judgement. We analyze image specificity with respect to image content and properties to better understand what makes an image specific. We then train models to automatically predict the specificity of an image from image features alone without requiring textual descriptions of the image. Finally, we show that modeling image specificity leads to improvements in a text-based image retrieval application.
Abstract (translated by Google)
对于一些图像，由多人写的描述是一致的。但对于其他图像，跨人的描述差异很大。换句话说，一些图像是特定的$ - $他们引起不同人的一致的描述$ - $而其他图像是不明确的。涉及图像和文字的应用程序可以通过了解哪些图像是特定的以及哪些图像是模糊的而受益。例如，考虑基于文本的图像检索。如果查询描述与模糊图像的标题（或参考描述）中等相似，那么该查询可以被认为是与图像相称的匹配。但是，如果图像非常具体，查询和参考描述之间的适度相似可能不足以检索图像。在本文中，我们介绍图像特异性的概念。我们提出了两种机制来衡量一个特定的图像的多重描述的特异性：一个自动化的措施和措施，依靠人的判断。我们分析图像内容和属性的图像特异性，以更好地了解是什么使图像具体。然后，我们训练模型，自动从图像特征自动预测图像的特异性，而不需要对图像进行文字描述。最后，我们展示了建模图像特异性导致基于文本的图像检索应用程序的改进。
URL
https://arxiv.org/abs/1502.04569