Active and passive forensic methods for protection of digital content

Abstract

With the development of display technology and computer graphics technology, various types of digital contents as well as 2D images and video have appeared. As a result, while the utilization of other types of content other than 2D image and video content has been low for a long time, new types of content such as 3D stereoscopic content and 3D model data have become highly utilized. Especially, as the range of usage of contents becomes wider and its influence becomes larger the importance of contents protection has been raised. Even though forensic techniques to protect digital content have been studied over the past decade, most have been studied exclusively in 2D video and video content. Also, the underlying technology has been limited to signal processing and image processing technologies. However,it is difficult to apply forensic technology based on 2D image and video contents directly to 3D stereoscopic contents or 3D model data due to differences in content characteristics. In order to develop forensic technology to protect specific contents, adaptive research on the contents is required, and a sufficient level of performance can be expected by including such a process. In addition, besides signal processing and image processing technology as base technology, it is possible to develop forensic technology with higher performance and adaptability by applying artificial intelligence technology, which has recently been appreciated. In dissertation, an active forensic technique for protection of 3D model data, a passive forensic technique for protection of 3D stereoscopic image, and a passive forensic technique based on artificial intelligence for 2D image protection are proposed. As an active forensic technology for protecting 3D model data, we propose watermarking technology robust against cropping attack of 3D model data. We also propose a 3D stereoscopic image adaptive resampling detection technique for detecting manipulation of 3D stereoscopic images. Finally, we propose a deep learning based detection technique to detect camera recapturing process and composite manipulation of 2D images. More specifically, the dissertation first introduces a 3D mesh watermarking technique robust to cropping attack through local shape-based synchronization. The proposed local shape-based synchronization technique is robust not only against to cropping attack but also against similarity attack and distortion attack because it uses shape of model rather than surface of model. In watermark embedding process, the distortion caused by watermark embedding is minimized by spreading using segmented bin of mesh. In addition, the proposed scheme has a higher level of security than the existing scheme. Experiments show that the proposed technique has high invisibility and high robustness against general signal processing attack and strong cropping attack. Second, this dissertation proposes a resampling detection method for stereoscopic images. Although previous resampling technique can be applied to stereoscopic images, performance improvement is hard to be expected with the two separated results. In this research, we found a strong relationship between the left and right images derived from the characteristics of the stereoscopic images. The proposed technique exploits that relationship of the stereoscopic images as additional information for reliable detection performance. Furthermore, the proposed method includes a preprocessing step to acquire the independent performance from the image's own characteristics. The experimental results exhibi superior performance compared with the existing works. Finally, dissertation proposes a re-capturing detection technique and composite manipulation detection technique based on deep learning. Conventional forensic techniques have used human-designed features to detect fingerprints for manipulation. On the other hand, deep learning technology automatically learns the feature points of an image. Also because the forensic problem eventually results in classification problems, it is suitable for applying the deep learning technique. Especially, when the deep learning technique is applied to the forensic problem, the adaptability of the technology is very high since a new type of manipulation can be responded through re-learning. In addition, when various manipulations are applied to an image at the same time, the manipulated images can be integrally learned and can be classified from a non-manipulated set. In this dissertation, we propose two techniques for detecting image recapturing and composite manipulation using convolutional neural network. Experimental results also demonstrate the high level of performance of those technologies.