Deep learning-based object detection of “Four-Panel Cartoon" in big data: object detection, data analysis of image-based cultural contents in digital documents, and exploration of practical applications

Abstract

In the realm of cultural content analysis and academic research, the collection of image-based cultural content or historical image objects from big data is a cardinal facet. However, this process is as intricate as extracting gold from vast terrains. Echoing this sentiment, there’s a rising tide of appreciation in scholarly circles for “Four-panel Cartoons” (FPCs) as a valuable cultural and historical content source in big data digital newspapers in Korea. Yet, akin to a treasure hunter’s endeavor, identifying these FPCs amidst the vastness of big data archives is an intricate journey, especially given their unstructured image data format — a task both time-intensive and costly. To address this issue, this research paper presents a novel computational data strategy: the development of the YOLOv5 FPC model, via fine-tuning the You Only Look Once Version 5 (YOLOv5) deep learning model, tailored precisely for FPC image detection. The original YOLOv5 model was fine-tuned using the “Four-Panel Cartoon Image Dataset,” [34] which we collected for optimization of the model. The resultant YOLOv5 FPC model showcased an F1-score of 0.97 for FPC detection, despite the small number of training data we were able to collect. When we applied our YOLOv5 FPC model to the Chosun Ilbo News Library [24] archive (1920-1940) for automated object detection, spanning 47,777 image files, we identified 1040 FPC objects within 1035 files, which include previously undiscovered FPCs by previous researchers, which we used for data analysis. These FPC images have been made available on the Harvard Dataverse website [32]. The seamless integration of the YOLOv5 FPC model with the Google Colab platform [33] offers users an intuitive platform for automated FPC detection within large digital collections, substantially reducing time, energy, and associated costs. This study presents an innovative methodology for detecting FPCs, offering new insights into cultural content and historical research.