WO2006076789A1

WO2006076789A1 - A bronchoscopy navigation system and method

Info

Publication number: WO2006076789A1
Application number: PCT/CA2006/000021
Authority: WO
Inventors: Doron Dekel
Original assignee: Claron Technology Inc.
Priority date: 2005-01-24
Filing date: 2006-01-09
Publication date: 2006-07-27

Abstract

A system and method for guiding a video bronchoscope placed into the lungs of a person towards a target to be observed or treated. The person is first scanned using a volumetric scanner, such as CT or MRI, to produce a volumetric (3-D) image data of their lungs. A skeleton representation of the bronchial tree is extracted and labeled. After an initialization step, incremental tracking of the pose of the bronchoscope's tip is performed by combining two different estimates of the change in the tip's pose relative to the volumetric data: (a) the longitudinal and rotational motion of the bronchoscope shaft; and (b) the results a registration between camera and volumetric data spaces using an iterative matching between the bronchoscopic video image and simulated virtual camera views generated from the volumetric data. The two estimates are combined based on an assessment of the reliability of the image registration results.

Description

A BRONCHOSCOPY NAVIGATION SYSTEM AND METHOD

The present invention relates to spatial guidance during medical procedures in which a flexible bronchoscope is used.

BACKGROUND OF THE INVENTION

[0001] Bronchoscopy aims to provide a minimally invasive access to one or more target locations in the lung, entering from the mouth or nose through the branching pulmonary airways. A target location, such as a nodule, is first identified in a CT or MRI scan and is often located at the periphery of the lung. The physician performing the procedure is first required to navigate the tip of the flexible endoscope through the maze-like bronchial tree to reach as close as possible to the target, and then to advance an instrument, such as a biopsy needle, to enter the target mass itself, at times without being able to observe it directly. These two tasks are very difficult in practice. There is, therefore, a need for a system to spatially guide the physician during such a procedure.

[0002] PCT patent WO03086498 to Gilboa describes a system that relies on a miniature electromagnetic sensor located at the distal tip of a special catheter inserted through the working channel of the bronchoscope to provide localization information. The position information provided by such a system is, however, subject to substantial errors, either due to the presence of nearby ferromagnetic objects, and/or mis-registration between the CT scan and the position measurement coordinate system as a result, for example, of tissue motion due to breathing, heartbeat or slight changes in body position.

[0003] Another approach is proposed by Higgins et al in the paper "Integrated Bronchoscopic Video Tracking and 3D CT Registration for Virtual Bronchoscopy". SPIE Proc. vol. 5031, 2003. In this system live video bronchoscopy images are iteratively matched with simulated (virtual) bronchoscopy views rendered from the CT slice stack data to estimate the motion of the bronchoscope from one instant to the other. When successful, this approach, called "optical flow tracking", provides a highly accurate positioning data directly in CT slice stack coordinates, eliminating the problems associated with either coordinate system mis-registration or external position measurement accuracy. Unfortunately, however, video images often become temporarily unsuitable for optical flow tracking due to a range of uncontrollable conditions, such as lens wash-out due to mucus or water, over- or underexposed frames, motion blur, appearance of obscuring bubbles or the disappearance of distinguishing features when the bronchoscope is aimed at a wall. If, as often happens, the bronchoscope is moved while optical flow tracking is thus disabled, position information is lost and cannot be easily restored.

[0004] The aim of the current invention is to provide a bronchoscopy navigation system to obviate or mitigate the disadvantages of the approaches presented above.

SUMMARY OF THE INVENTION

[0005] The present invention provides a tracking system and method for obtaining a mapping between the coordinate system of a video image obtained from a bronchoscope and the coordinate system of a stack of slice images obtained from the patient undergoing the bronchoscopy-guided procedure.

[0006] A computer processes the slice images to extract a model of the patient' s bronchial tree, and then to segment and uniquely label each non-branching segment of the tree.

[0007] A movement sensor is placed in a position substantially fixed relative to the endoscope's entry point into the patient's head to detect in/out and roll motion of the bronchoscope. The movement data from the sensor is fed to the tracking computer, together with the stream of video images from the bronchoscope.

[0008] In a tracking initialization step, the bronchoscope is placed approximately at a known position and orientation within the bronchial tree, providing a known current tree segment identifier and an approximate position coordinates and aim angles along the segment. The computer then iteratively adjusts the estimated pose (position and aim) to improve the match between the video image and a simulated "virtual bronchoscopy" image generated form the slice data at the adjusted pose, to obtain a more accurate mapping between the two coordinate systems.

[0009] Using the mapping thus obtained, the computer uses the tree model to label each branch entry point visible in the video image. As the bronchoscope is moved to enter a new tree segment, the current segment identifier and the image pose in the segment are accordingly updated.

[00010] During in/out or roll movement along a tree segment, the movement sensor and, when possible, the iterative video image registration step described above, allow the pose of the bronchoscope image to be updated, ensuring uninterrupted tracking.

[00011] The coordinate mapping between the slice stack coordinates and the image coordinates provided by the tracking system may be used to provide the user with visual navigation guidance indicators, such as graphical indicators depicting the planned path and/or the target region laid over the bronchoscope's video image.

BRIEF DESCRIPTION OF THE DRAWINGS

[00012] These and other features of the invention will become more apparent from the following description in which reference is made to the appended drawings wherein:

[00013] Figure 1 shows the components of the bronchoscopy tracking system. [00014] Figure 2 shows the main steps involved in the tracking process.

DESCRIPTION OF PREFERRED EMBODIMENT Introduction

[00015] The bronchial tree starts with the trachea at its root, splitting into progressively narrower bronchi at branching points. The pipe-like geometry of each bronchus can be usefully approximated by its medial line, and the tree thus approximated by a skeleton made of non-branching line segments interconnected at the branching points. The location of the bronchoscope's tip can be correspondingly usefully described by only two coordinates: the identifier of the specific tree segment, eg, an arbitrary serial number, and the distance along the segment from the segment's branching point closer to the root. These coordinates are unaffected by patient and tissue movements.

[00016] Let us further define, for each location along the tree segment, a segment coordinate system with its origin at that location, its Z axis extending in the direction of the skeleton line's tangent away from the root, and its X axis extending radially in a direction defined in reference to sibling branches. For example, at the start of the segment, the X axis may be defined as the projection of the lowest-numbered sibling branch on the radial plane. At other locations along the segment, it can be defined as the radial vector forming the smallest angle with the X axis vector at the start.

[00017] Let us also define an image coordinate system formed by the lens-sensor assembly mounted at the tip of the endoscope. This coordinate system maps each pixel location to a projection line in space relative to the bronchoscope's tip location and shaft vector orientation.

[00018] Providing navigation assistance during bronchoscopy preferably includes a visible indication of the geometrical relationship between the current pose of the bronchoscope' s tip and either a pre-planned path to a desired tip pose or one or more locations of interest indicated in the pre-acquired CT volume. This can preferably be provided to the user in the form of a graphic overlay over the bronchoscopic video image, or as a highlighted location on a simulated external view of the tree, or both. Solution outline

[00019] From the above discussion, the task of providing the bronchoscopy navigation guidance is translated into the following more specific tasks.

1. Calibrate the tip camera to derive a bronchoscope tip pose to image space mapping. This needs to be done only once for each bronchoscope. 2. Extract a tree skeleton from the patients' CT study, identifying and uniquely labeling all its segments in the process.

3. Allow the user to mark a path and/or target locations/regions for the bronchoscopy- based procedure in the CT slice stack coordinate space (optional).

During the procedure itself, periodically: 4. Identify the tree skeleton segment in which the bronchoscope's tip currently resides.

5. Measure the pose of the image coordinate system in the tree segment's coordinates using a combination of video image matching and sensing of the longitudinal and rotational motion of the endoscope since its last known pose.

6. Use the above measurements and the tip pose to image space mapping to: a. map the planned path and/or target locations from CT study space to image space and show them as video image overlay; and/or b. map the tip location and orientation to CT study space to overlay an indicator on an image of the bronchial tree model. System overview

[00020] Referring to figure 1, the system consists of a commonly available flexible bronchoscope 10 containing a miniature video camera at its tip 20. The bronchoscope is connected to its controller 15. The controller provides a video output 30 containing a signal that may be displayed on a video monitor and/or, as shown, connected to a video input of computer 50. The bronchoscope shaft 12 may contain one or more channels to allow fluids or instruments to be sent to its tip. Computer 50 is provided with persistent storage unit 48, onto which a stack of CT slices of the lung 5 acquired from patient 1 is loaded.

[00021] Rigid mask 70 is placed over the patient's mouth and nose and secured using a head strap. The mask 70 is perforated to allow the patient to breath normally. A movement sensor assembly 75 is fastened to the front of the mask over an opening, allowing shaft 12 of bronchoscope 10 to be inserted into the patient's nose or mouth. In alternative embodiments, the sensor may be fastened to other types of artifacts, such as a mouthpiece, expected to stay in a substantially fixed position in relation to the patient's skull. Movement sensor assembly 75 sends its output signals to computer 50, allowing the computer to compute the amount of in/out and clockwise/counterclockwise movement of the bronchoscope relative to the sensor's housing.

[00022] Sensor assembly 75 may be constructed in one of many known methods for sensing X/Y movement, such as those being used in various types of computer mouse design. Tangential movement of the bronchoscope's shaft can be easily converted to roll angle using the diameter of the bronchoscope's shaft, which can either be manually entered into the system by the user, or automatically sensed by another sensor in the assembly.

[00023] Referring to the list of tasks outlined above, systems and methods for accomplishing tasks 1, 3 and 6 are well known in the art and will not be further explained herein. Preferred methods for accomplishing the other tasks are further described below. Extracting and labeling a bronchial tree skeleton (task 2)

[00024] A number of algorithms for tree extraction from CT lung scans are already known, such as the one described in the paper "Hybrid Segmentation and Exploration of the Human Lungs" by Dirk Bartz et al, in the proceedings of IEEE Visualization 2003. Converting an extracted tree to a medial line representation is also well known using morphological thinning operations. Finally, uniquely identifying each line segment and branching point is straightforward. For example, an ascending serial number may be assigned to each line segment in the order it is detected, and a branching point may be uniquely identified by the sorted list of identifiers of the line segments meeting at the point, or by using an explicit representation of the connectivity graph.

Tracking the bronchoscope's tip pose during the procedure (tasks 4 and 5)

[00025] At the start of the procedure, segment identification is trivial, since the bronchoscope's tip is always inserted through the trachea. Referring to figure 2, in tracking initialization step 95, the user signals the computer when the tip approaches the first carina separating the left and the right mainstem bronchi (LMSB/RMSB), and the image is clear and in a specified orientation (eg, LMSB on the left and RMSB on the right).

[00026] Starting from a default initial pose estimate in front of the first carina, an iterative matching algorithm is executed in step 100 to adjust the estimated pose to register the actual video image with a virtual video image generated from the CT slice stack A suitable iterative matching algorithm is described in the paper "New Image Similarity Measure for

Bronchoscope Tracking Based on Image Registration" by Deguchi et al, Proceedings of MICCAI 2003, p. 399-406 and others papers referenced thereby. At the completion of this process, the camera pose along the bronchial tree is correctly established.

[00027] Using the computed pose along the tree skeleton, the segments branching out forward can now be projected and overlaid, correctly labeled, over the registered video image in step 110. The planned path towards the target, if one was prepared, can be highlighted, and the direction to the (possibly hidden) target region may be shown.

[00028] In subsequent step 120 the pose is compared with the coordinates of the current and the forward branching segments to detect when a new segment has been entered. If such is the case, step 130 is executed to update the current segment identification and the pose in the new segment's coordinate system.

[00029] In step 140, signals from movement sensor 75 are evaluated to detect the amount of in/out movement and clockwise/counterclockwise roll of the bronchoscope shaft as it moves away from its known pose. Once a movement is detected, its magnitude is used to generate a new pose estimate for the tip.

[00030] A test is performed in step 150 to establish whether video images are clear and show a sufficient amount of useful features. If so, steps 100 and 110 are executed to refine the pose estimate and label visible bronchus entry points, if any. The new pose estimate obtained from the sensed movement of the bronchoscope's shaft is used to initialize the registration process. Compared to the registration process known in prior art where the iterative search for best image match does not have a reliable initialization estimator, the more accurate estimation generated from the sensor measurements provides increased robustness and efficiency to the registration algorithm. If the test in step 150 indicates that the video image is unsuitable for registration, step 140 is executed again.

In case step 140 indicates that the bronchoscope has advanced beyond the termination point of the current segment, but without obtaining an indication of the subsequent segment entered, the system may issue a visible or audible warning to the user. In such a case, the user may retreat the bronchoscope slightly and wait until the system indicated it was able to execute steps 100 and 110 successfully.

Claims

WHAT IS CLAIMED IS:

1) A method for providing assistance in navigating the tip of a flexible video bronchoscope within the bronchial tree of a person, the method characterized by including the steps of: a) Measuring the longitudinal and rotational motion of the bronchoscope shaft to obtain a first estimate of the pose of the bronchoscope's tip relative to said bronchial tree; b) Registering the broncho scopic video images to simulated bronchoscopic images derived from volumetric data acquired from said person to obtain a second estimation of the pose of the bronchoscope' s tip; and c) Combining, or selecting between, the two abovementioned pose estimates to generate a final pose estimate

2) The method of claim 1 , further characterized by comprising the task of assessing the quality of the bronchoscope video image and using the results of this assessment in determining how to combine or select between said first and second pose estimates.

3) The method of claim 1 , wherein said first estimate is used when the step of generating said second estimate fails.

4) The method of claim 1, wherein the measurement of the bronchoscope's motion is done relative to an object held in a substantially fixed relation to said person's head. 5) The method of claim 4, wherein said object is a mask placed over said person's nose or mouth.

6) The method of claim 4, wherein said object is a mouthpiece.

7) The method of claim 1 , further characterized by including the steps of marking a target location in said volumetric data and using the combination of said target location and said final bronchoscope tip pose estimate to provide additional navigational assistance.

8) The method of claim 7, wherein said additional navigational assistance includes providing graphical marks on the bronchoscopic video image indicating the projection of said target location on said video image.

9) A method for providing assistance in navigating the tip of a flexible video bronchoscope within the bronchial tree of a person, the method characterized by including the steps of: a) Using an iterative matching algorithm to register the bronchoscopic video images to simulated bronchoscopic images derived from volumetric data acquired from said person, thereby obtaining an estimation of the pose of the bronchoscope's tip within said volumetric data at time instance tθ; b) Obtaining measurements of the longitudinal and rotational motion of the bronchoscope shaft between time instance tO and a subsequent time instance tl ; and c) Using said shaft motion measurements to provide an initial pose estimate for the application of an iterative image matching algorithm to register the bronchoscopic video images taken at time tl to simulated bronchoscopic images derived from said volumetric data, thereby obtaining a more reliable estimation of the pose of the bronchoscope' s tip within said volumetric data at time instance tl;

10) The method of claim 9, wherein said measurements of said shaft motion are performed relative to a mask placed over said person's nose or mouth.

11) The method of claim 9, further characterized by including the steps of marking a target location in said volumetric data and using the combination of said target location and said final bronchoscope tip pose estimate to provide additional navigational assistance.

12) The method of claim 11, wherein said additional navigational assistance includes providing graphical marks on the bronchoscopic video image indicating the projection of said target location on said video image.