Facial expression based nuclear accident diagnosis performance estimation using a deep learning network

Abstract

In the industrial environment of today, most tasks are carried out by machines, but humans are involved in various aspects of industrial processes, from design to operation. Yet, quantitative research on the human factor in accident causation in consideration of cognitive human error is rarely reported. In this study, we take the view that the analysis of immediate facial expressions in real-time provides a non-intrusive estimate of human performance and improves the safety of the operation of industrial facilities. Therefore, we conducted experiments to estimate human performance based on facial expressions. The experiments demonstrated the potential use of inter-problem (up to 76%) and inter-person (up to 80%) facial expression for performance estimation, and even in real-time situations. Our results provide strong support for previous findings that considered facial expression as biological stressor responses and emotional states as related to performance impairing stress. We conclude that the time sequence of facial expression analysis can estimate human performance, and we foresee the extension of our approach to a more complex industrial field.