Indoor and outdoor illumination estimation from a single image using deep learning

Abstract

Unlike research on deep learning methods limited to the existing image environment (indoor or outdoor), this study aims to estimate suitable light sources from both indoor and outdoor environment images through one deep learning method. The network configuration consists of two steps: the Crop-to-PanoLDR network that estimates lighting conditions suitable for indoor or outdoor scenes from a single image as LDR environment maps, and the LDR-to-HDR network creates HDR environment maps which contain light information from LDR environment maps. Through such a process, the HDR environment map generated from a single image is applied when rendering a virtual object in a virtual environment to check the direction of light and ambient light. In order to verify the Crop-to-PanoLDR network, we trained this network with only the indoor scene images, only outdoor scene images and both indoor and outdoor scene images then compared the results. In addition, the effect of the presence of a classification head that classifies indoor and outdoor scenes in the Crop-to-PanoLDR network on training results was evaluated. Also, a user test was conducted to compare the estimated HDR environment map with the existing research results.