In this paper, we propose a novel autonomous driving system using sensor fusion odometry and real-time path planning which works robustly in rough field environments. The proposed method can perform a specific mission and return to the starting point while performing a mission. A robust sensor fusion-based odometry was developed for the target environment, and a real-time path planning was performed using an open map-based global cost map and a local cost map generated by traversable ground and above-ground object segmentation. The integrated system was tested in real rough terrains. The result of the sensor fusion odometry showed more accurate position estimation than the existing LiDAR inertial odometry (LIO) algorithm.