It is recognized that an abnormal situation diagnosis is a challenging task for nuclear power plant (NPP) operators because of the excessive information and high workload in such situations. To help operators, several studies have proposed operator support systems using artificial intelligence techniques. However, those methods could neither assess anonymous cases as an unknown situation nor confirm whether its outputs are reliable or not. In this study, an algorithm that can confirm the diagnosis results and determine unknown situations using long short-term memory (LSTM) and variational autoencoder (VAE) is proposed. LSTM was adopted as the primary network for diagnosing abnormal situations. Meanwhile, VAE-based assistance networks were added to the algorithm to ensure that the credibility of the diagnosis is estimated via the anomaly score-based negative log-likelihood. The algorithm was tested and implemented using the compact nuclear simulator for the Westinghouse 900 MWe NPP. Furthermore, the simulation also considered noise-added data. (C) 2020 Elsevier Ltd. All rights reserved.