The use of machine learning (ML) is exploding in materials science as a result of its high predictive performance of material properties. Tremendous trainable parameters are required to build an outperforming predictive model, which makes it impossible to retrace how the model predicts well. However, it is necessary to develop a ML model that can extract human-understandable knowledge while maintaining performance for a universal application to materials science. In this study, we developed a deep learning model that can interpret the correlation between surface electronic density of states (DOSs) of materials and their chemisorption property using the attention mechanism that provides which part of DOS is important to predict adsorption energies. The developed model constructs the wellknown d-band center theory without any prior knowledge. This work shows that human-interpretable knowledge can be extracted from complex ML models.