Image-based global registration system and method applicable to bronchoscopy guidance

• US 10,667,679 B2
• Filed: 03/22/2018
• Issued: 06/02/2020
• Est. Priority Date: 01/28/2010
• Status: Active Grant

• Current Assignee: Penn State Research Foundation (Pennsylvania State University)
• Original Assignee: Penn State Research Foundation (Pennsylvania State University)
• Inventors: Higgins, William E., Khare, Rahul, Merritt, Scott A.
• Primary Examiner(s): Neal, Timothy J
• Assistant Examiner(s): Woo, Jae
• Application Number:
  US15/928,974
• Publication Number:
  US 20180220883A1
• Time in Patent Office: 803 Days
• Field of Search: 600117
• CPC Class Codes: A61B 1/00009 of image signals during a use of endoscope
• CPC Class Codes: A61B 1/2676 Bronchoscopes
• CPC Class Codes: G06T 2207/10068 Endoscopic image
• CPC Class Codes: G06T 2207/30061 Lung
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06V 10/754 involving a deformation of the sample pattern or of the reference pattern; Elastic matching
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects
• CPC Class Codes: G06V 20/653 by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces

Abstract

A global registration system and method identifies bronchoscope position without the need for significant bronchoscope maneuvers, technician intervention, or electromagnetic sensors. Virtual bronchoscopy (VB) renderings of a 3D airway tree are obtained including VB views of branch positions within the airway tree. At least one real bronchoscopic (RB) video frame is received from a bronchoscope inserted into the airway tree. An algorithm according to the invention is executed on a computer to identify the several most likely branch positions having a VB view closest to the received RB view, and the 3D position of the bronchoscope within the airway tree is determined in accordance with the branch position identified in the VB view. The preferred embodiment involves a fast local registration search over all the branches in a global airway-bifurcation search space, with the weighted normalized sum of squares distance metric used for finding the best match.

Claims

1. A global registration system useful in bronchoscopic guidance and other applications, comprising:

a memory storing virtual bronchoscopy (VB) renderings of a 3D airway tree, the renderings including VB views and branch positions within the airway tree;

a computer operative to search and analyze the VB renderings to identify the VB view and branch position that most closely matches a real bronchoscopic (RB) view from a bronchoscope of an unknown or arbitrary location along an unknown or arbitrary airway branch of the airway tree; and

wherein the computer is operative to perform a search across at least a plurality of airway branches in the airway tree and wherein the search comprises;

an intra-branch search within each airway branch of the plurality of airway branches of the airway tree to identify an optimal VB view along each airway branch of the plurality of airway branches of the airway tree that matches the received RB view, wherein the intra-branch search within each branch scans through a plurality of pre-defined view positions in that branch; and

an inter-branch search subsequent to the intra-branch searches in which at least one of the optimal VB views from all of the intra-branch searches that most closely matches the RB view is identified and used for determining a 3D position of the bronchoscope; and

a display presenting the 3D position of a bronchoscope within the airway tree based upon the VB view and branch position identified in the VB view.

2. The system of claim 1, wherein the computer is operative to identify a plurality of most likely branch positions having VB views closest to the received RB view.

3. The system of claim 1, wherein the computer uses domain-specific information regarding general bronchoscope position within the airway tree.

4. The system of claim 3, wherein the domain-specific information includes current lung designation or lung lobar position.

5. The system of claim 1, wherein the computer is operative to identify an optimum branch position having a VB view closest to each of a plurality of RB views, and wherein the multiple RB views are associated with the same or multiple branch positions within an airway tree.

6. The system of claim 1, wherein the intra-branch search performed by the computer comprises a fast search over all the branches in a global airway-bifurcation search space using a weighted normalized sum of squares distance metric to determine a best match.

7. The system of claim 1, wherein the intra-branch search performed by the computer comprises using pre-computed lumen region enclosing rectangles and a fast local registration refinement.

8. The system of claim 1, wherein the intra-branch search performed by the computer is carried out along a centerline of each airway branch.

9. The system of claim 1, wherein the matched VB view and RB view are correlated to a bifurcation feature.

10. The system of claim 1, wherein the computer is further operative to match the VB view and the RB view along the airway branches including portions of the airway branches between bifurcations.

11. The system of claim 7, wherein a best rendering position is provided corresponding to each optimal view from each intra-branch search and different roll angles are applied to the best rendering position to obtain different rendering positions, and wherein the local registration refinement includes adjusting the different rendering positions using an inverse compositional method for local registration.