Content-aware image resizing is the process of adjusting the size of an image while preserving its important content. Image resizing is used to overcome diversity in resolutions between modules, such as display devices and applications, and can thus be deliberately exploited to distort or remove original content; therefore, detecting such tampering has become an important topic in forensics. This paper proposes a deep neural network architecture to capture subtle local artifacts caused by seam-based image resizing. Unlike past approaches that only classified two classes, our approach is the first attempt to solve a given forensic task with three-class classification: original, seam insertion, and seam carving. The experimental results show that our work performs better than the handcrafted feature-based method and networks designed for different forensic tasks.