This study addresses vision-based underwater navigation techniques to automate underwater intervention tasks with robotic vehicles. A systematic procedure of model-referenced pose estimation is introduced to obtain the relative pose information between the underwater vehicle and the underwater structures whose geometry and shape are known. The vision-based pose estimation combined with inertial navigation enables underwater robots to navigate precisely around underwater structures for challenging underwater intervention tasks such as subsea construction, maintenance, and inspection. To demonstrate the feasibility of the proposed approach, a set of experiments were carried out in a test tank using an autonomous underwater vehicle.