See, caption, cluster: Large-scale image analysis using captioning and topic modeling

Cited 0 time in webofscience Cited 0 time in scopus
  • Hit : 337
  • Download : 0
Owing to the widespread use of smartphones and mobile devices and the prevalence of image-sharing social network services, the amount of image data available on the Web is soaring. Various tasks, such as image classification, detection, and segmentation, use tremendous amounts of image data to train machine learning models. Using these trained models, a visual feature representation vector can be extracted from individual images and subsequently be used in several applications, such as image retrieval, object detection, and clustering. However, despite the increasing demand for such analyses, few studies have analyzed the information summarized by such image datasets, especially for extracting topics, trends, and opinions from images generated by online communities. Therefore, we propose a novel approach to image topic modeling, which accounts for visual content as well as semantic information by leveraging the image captioning model. In addition, we propose an image-caption scoring model that measures the semantic similarity between an image and its generated caption in order to filter noisy data that obstruct analysis by obscuring the semantic meaning of topics extracted from the dataset. The results show that our proposed method assists in analyzing large-scale image datasets without the need to manually check individual images. Further experimental results show that our methods are particularly beneficial for applications such as data visualization, image retrieval, and image tag recommendation in the realm of large-scale image dataset analysis.
Publisher
PERGAMON-ELSEVIER SCIENCE LTD
Issue Date
2024-03
Language
English
Article Type
Article
Citation

EXPERT SYSTEMS WITH APPLICATIONS, v.237

ISSN
0957-4174
DOI
10.1016/j.eswa.2023.121391
URI
http://hdl.handle.net/10203/313892
Appears in Collection
AI-Journal Papers(저널논문)
Files in This Item
There are no files associated with this item.

qr_code

  • mendeley

    citeulike


rss_1.0 rss_2.0 atom_1.0