With the development of data collection technology, detecting anomalies in a large amount of time-series data with an automated system is challenging. Identifying properties using trends and periodicity through time-series decomposition helps to figure out complex time-series patterns. However, previous approaches for detecting anomalies in time series did not take into account temporal auxiliary information such as holidays, limiting their ability to respond exceptionally to unusual circumstances. For example, a sharp increase in the value of a given variable might be normal on holidays but anomalous on weekdays. In this study, we propose a framework that helps to detect anomalies through time-series decomposition based on a deep neural network by exploiting temporal auxiliary information. Through experiments on the real-world dataset and public referenced datasets, we show that the anomaly detection using the residuals of context-based decomposition improves performance by up to 37% in conventional metrics and 49% in time-series aware metrics compared with existing methods.