DC Field | Value | Language |
---|---|---|
dc.contributor.advisor | Kweon, In So | - |
dc.contributor.advisor | 권인소 | - |
dc.contributor.author | Choi, Jinsoo | - |
dc.date.accessioned | 2021-05-12T19:41:13Z | - |
dc.date.available | 2021-05-12T19:41:13Z | - |
dc.date.issued | 2020 | - |
dc.identifier.uri | http://library.kaist.ac.kr/search/detail/view.do?bibCtrlNo=909453&flag=dissertation | en_US |
dc.identifier.uri | http://hdl.handle.net/10203/284216 | - |
dc.description | 학위논문(박사) - 한국과학기술원 : 전기및전자공학부, 2020.2,[xi, 103 p. :] | - |
dc.description.abstract | Recently, various social networking services (SNS) and video platforms have gathered much video data. There have been reports claiming approximately 400 hours worth of videos being uploaded to the web every minute, that is leading to an estimation that videos will take over 82% of the Internet traffic by 2022. Due to this phenomenon, many companies are focusing on video related applications and services. In this era of abundant video data, unsupervised learning from videos are gathering much attention as well. We propose the following research on video frame interpolation for stabilization and super resolution. First, we introduce full-frame video stabilization via iterative frame interpolation. Video stabilization is a fundamental and important technique for higher quality videos. Prior works have extensively explored video stabilization, but most of them involve cropping of the frame boundaries and introduce moderate levels of distortion. We present a novel deep approach to video stabilization which can generate video frames without cropping and low distortion. The proposed framework utilizes frame interpolation techniques to generate in-between frames, leading to reduced inter-frame jitter. Once applied in an iterative fashion, the stabilization effect becomes stronger. A major advantage is that our framework is end-to-end trainable in a self-supervised manner. We show the advantages of our method through quantitative and qualitative evaluations comparing to the state-of-the-art methods. Second, we propose a real-time video stabilization method via self-supervised learning. Unlike the majority of methods that run offline, our approach is designed to run in real-time. Our framework consists of a rigid transformation estimation between given frames for global stability adjustments, followed by scene parallax reduction via smoothed flow for further stability. Then, an inpainting module fills in the missing margin regions created during stabilization, in order to reduce the amount of margin cropping as post-processing. These sequential steps reduce distortion and margin cropping to a minimum while enhancing stability. Hence, our approach outperforms state-of-the-art methods as well as offline methods that require camera trajectory optimization. Third, we propose a joint super resolution (SR) and frame interpolation algorithm which can be applied to video reconstruction and compression. Previous works have addressed super resolution and frame interpolation as separate topics. Although significant advances have been made for each field of research, the joint task has not been explored extensively. A naive solution of applying SR and frame interpolation (and in reverse order) is sub-optimal. We propose a joint SR and frame interpolation model via permutation invariance. Our approach demonstrates favorable results to sequential application of state-of-the-art SR and frame interpolation methods. Lastly, we conduct research on solving the limitations of frame interpolation. The remaining challenges for frame interpolation are fast object motion and nonlinear motion. In this work, we address the nonlinear motion via modeling with three input frames instead of the typical two frame input. As for fast object motion, we take a data-driven approach by augmenting data samples on-the-fly via adding objects to the video frames conveying flying motion. In the appendix, we propose video summarization via natural language and the problem of composing a story out of multiple short video clips. These tasks use progressive learning and optimization techniques respectively. | - |
dc.language | eng | - |
dc.publisher | 한국과학기술원 | - |
dc.subject | Deep learning▼aSelf-supervised learning▼aVideo stabilization▼aFrame interpolation▼aSuper resolution | - |
dc.subject | 심층학습▼a자기지도학습▼a비디오 안정화▼a영상 보간법▼a초해상도 | - |
dc.title | Deep learning approaches for video frame interpolation and super resolution | - |
dc.title.alternative | 심층학습을 통한 영상 보간법 및 초해상도 기술 | - |
dc.type | Thesis(Ph.D) | - |
dc.identifier.CNRN | 325007 | - |
dc.description.department | 한국과학기술원 :전기및전자공학부, | - |
dc.contributor.alternativeauthor | 최진수 | - |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.