A new method with an artificial intelligence technique for vision-based navigation is proposed in this paper. In order to overcome the vulnerabilities of GPS/INS such as jamming and spoofing, vision-based navigation has been widely researched. However, most of the methods for vision-based navigation did not consider the difference between the scene and the database. Therefore, we adopted the semantic segmentation to extract the time-invariant features in the aerial imagery and approached the matching problem as comparing two Gaussian mixture models. Based on L2 distance between two Gaussian mixture models, the particle filter estimates the region most similar to the aerial imagery. For demonstrating the performance, the simulation is implemented in the terrain with dense and sparse features, respectively. Furthermore, the particle filter estimated the true trajectory within the horizontal error of 10m a regardless of the terrain.