Feature-based camera tracking for 3D-to-2D image registration for image-guided endoscopic retrograde cholangiopancreatography

Abstract

Endoscopic Retrograde Cholangiopancreatography (ERCP) is a procedure used to diagnose and treat problems in bile duct or pancreatic duct. One of the most difficult tasks in the procedure is when the tip of endoscope enters the duodenum, where the surgeon needs to insert a catheter through the endoscope into the bile duct or pancreatic duct inside the major duodenal papilla. It is very difficult for the surgeon to insert catheter into the duct system in correction direction because the limited view of endoscopic camera. The surgeon relies almost on the endoscopic camera view to find the exact position of the major papilla and based on experience he determines the direction to insert the catheter into the common duct system through the major papilla. Furthermore the blood vessels under the surface of duodenum can be easily damaged when the surgeon tries to cut into the major papilla. So it would be helpful if the doctor can see duct system and blood vessels in endoscopic image. This paper proposes a method for registering 3D organ models on 2D endoscopic image during intraoperative ERCP procedure. To register 3D organs model on 2D endoscopic image, the relative position and orientation of endoscopic camera in the organs coordinate system need to be known because a virtual camera need to follow the real endoscopic camera to render these 3D model on top of endoscopic image correctly. The method proposed in this paper is a feature-based camera tracking method. The user is required to choose one first reference frame in the endoscopic video, and provide some correspondences between a virtual image of major papilla and this reference frame to initialize the camera pose of the first chosen frame. The first chosen frame then becomes a reference frame. The tracking then is done automatically in the following frames. After the camera tracking is triggered, the current camera frame is compared with the first reference frame and the previous frame to find the corresponden...