Domain-Specific Image Caption Generator with Semantic Ontology

Abstract

Image captioning is the task of generating textual descriptions of a given image, requiring techniques of computer vision and natural language processing. Recent models have utilized deep learning techniques for this task to gain performance improvement. However, these models can neither fully use information included in a given image such as object and attribute, nor generate a domain-specific caption because existing methods use open dataset such as MSCOCO which include general images. To overcome these limitations, this paper proposes a domain-specific image caption generator, which generates a caption based on attention mechanism with object and attribute information, and reconstruct a generate caption using a semantic ontology to provide natural language description for given specific-domain. To show the effectiveness of the proposed model, we evaluate the image caption generator with a dataset, MSCOCO, quantitatively and qualitatively.