Semantic segmentation of urban scenes using spatio-temporal contexts

Abstract

This dissertation proposes a novel method for the joint inference of road layouts and the semantic segmentation of urban scenes using spatial and temporal contexts. An urban city is a space where the moving objects obey traffic rules, and all the objects are arranging on the space based on their function and purpose. It means that the movement of dynamic objects and the locational preferences in the urban space lie within a predictable range. The proposed method aims to improve the semantic segmentation performance of urban scenes by modeling these spatial and temporal contexts. First, a special road coordinate system is defined to model spatio-temporal relations that can be obtained by inferring the parameterized road layout of urban scenes. The proposed relation models are applied to the pixels in which the depth information is valid. We designed two spatial contexts as location priors and label co-occurrences, and the temporal context as object temporal priors from the estimated scene flow. All the contexts are defined on the additional potential functions of the conditional random field model. The proposed method is validated with the various publicly available urban datasets including images and the corresponding depth measurements.