Separable explanations of deep neural network decisions

Abstract

In this thesis we examine an effect that occurs when applying Deep Taylor Decomposition to Neural Networks.Deep Taylor Decomposition is a method to explain decisions of Neural Networks by mapping the output decision back onto the input. If it is applied to a non-dominant class, the resulting heatmap looks identical to the heatmap for the dominant class with the highest output. Therefore it is not a meaningful explanation of the non-dominant class. We examine this diffusion of different explanations by training Neural Networks with varying numbers of classes and varying networks structures and identify two potential causes. Subsequently we explore and analyze strategies to counter these causes and reduce diffusion. All strategies follow one of two approaches. Either the structure of the Neural Networks is changed before training to encourage separation of heatmaps or the relevance propagation rules are changed. To objectively rate and compare the validity of those strategies we introduce a metric to measure the quantity of diffusion present and examine its validity when compared to visible differences of the explanation. The metric alongside exemplary images is used to judge the success of mitigation strategies. For large numbers of classes, we found no efficient way to reduce diffusion by changing the network structure. Changed relevance propagation rules can be applied and lead to sensible explanations. For small numbers of classes to be classified, it is possible to either change the network structure without large losses in performance or training cost to acquire sensible explanations for non-dominant classes.