Abstract
Significant performance gains in deep learning coupled with the exponential growth of image and video data on the Internet have resulted in the recent emergence of automated image captioning systems. Ensuring scalability of automated image captioning systems with respect to the ever increasing volume of image and video data is a significant challenge. This paper provides a valuable insight in that the detection of a few significant (top) objects in an image allows one to extract other relevant information such as actions (verbs) in the image. We expect this insight to be useful in the design of scalable image captioning systems. We address two parameters by which the scalability of image captioning systems could be quantified, i.e., the traditional algorithmic time complexity which is important given the resource limitations of the user device and the system development time since the programmers’ time is a critical resource constraint in many real-world scenarios. Additionally, we address the issue of how word embeddings could be used to infer the verb (action) from the nouns (objects) in a given image in a zero-shot manner. Our results show that it is possible to attain reasonably good performance on predicting actions and captioning images using our approaches with the added advantage of simplicity of implementation.
Abstract (translated by Google)
在深度学习方面显着的性能提高，加上互联网上图像和视频数据的指数级增长，导致最近出现了自动图像字幕系统。确保自动图像字幕系统在不断增加的图像和视频数据量方面的可扩展性是一个重大的挑战。本文提供了一个有价值的见解，即图像中几个重要（顶部）对象的检测允许提取其他相关信息，如图像中的动作（动词）。我们期望这种见解在可缩放的图像字幕系统的设计中非常有用。我们解决了图像字幕系统的可扩展性可以量化的两个参数，即传统的算法时间复杂度，考虑到用户设备的资源限制和系统开发时间，因为程序员的时间是一个关键的资源约束许多现实世界的场景。此外，我们解决了如何利用词嵌入来以零点方式从给定图像中的名词（对象）推断动词（动作）的问题。我们的研究结果表明，使用我们的方法可以在预测动作和字幕图像方面取得相当好的性能，并具有实施简单的优点。
URL
https://arxiv.org/abs/1606.01393