IMAGE-BASED GLOBAL REGISTRATION SYSTEM AND METHOD APPLICABLE TO BRONCHOSCOPY GUIDANCE

• US 20180220883A1
• Filed: 03/22/2018
• Published: 08/09/2018
• Est. Priority Date: 01/28/2010
• Status: Active Grant

• Current Assignee: Penn State Research Foundation (Pennsylvania State University)
• Original Assignee: Penn State Research Foundation (Pennsylvania State University)
• Inventors: Higgins, William E., Khare, Rahul, Merritt, Scott A.
• Granted Patent: US 10,667,679 B2
• Time in Patent Office: Days
• CPC Class Codes: A61B 1/00009 of image signals during a use of endoscope
• CPC Class Codes: A61B 1/2676 Bronchoscopes
• CPC Class Codes: G06T 2207/10068 Endoscopic image
• CPC Class Codes: G06T 2207/30061 Lung
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06V 10/754 involving a deformation of the sample pattern or of the reference pattern; Elastic matching
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects
• CPC Class Codes: G06V 20/653 by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces

Abstract

A global registration system and method identifies bronchoscope position without the need for significant bronchoscope maneuvers, technician intervention, or electromagnetic sensors. Virtual bronchoscopy (VB) renderings of a 3D airway tree are obtained including VB views of branch positions within the airway tree. At least one real bronchoscopic (RB) video frame is received from a bronchoscope inserted into the airway tree. An algorithm according to the invention is executed on a computer to identify the several most likely branch positions having a VB view closest to the received RB view, and the 3D position of the bronchoscope within the airway tree is determined in accordance with the branch position identified in the VB view. The preferred embodiment involves a fast local registration search over all the branches in a global airway-bifurcation search space, with the weighted normalized sum of squares distance metric used for finding the best match.

Claims

1. A global registration system useful in bronchoscopic guidance and other applications, comprising:

a memory storing virtual bronchoscopy (VB) renderings of a 3D airway tree, the renderings including VB views and branch positions within the airway tree;

a computer operative to search and analyze the VB renderings to identify the VB view and branch position that most closely matches a real bronchoscopic (RB) view from a bronchoscope of an unknown or arbitrary location along an unknown or arbitrary airway branch of the airway tree; and

wherein the computer is operative to perform a search across at least a plurality of airway branches in the airway tree and wherein the search comprises;

an intra-branch search to identify an optimal VB view along each airway branch of the plurality of airway branches of the airway tree that matches the received RB view, wherein the intra-branch search spans less than all possible view positions in each branch; and

an inter-branch search subsequent to the intra-branch search in which at least one of the optimal VB views from the intra-branch search that most closely matches the RB view is identified and used for determining a 3D position of the bronchoscope; and

a display presenting the 3D position of a bronchoscope within the airway tree based upon the VB view and branch position identified in the VB view.

2. The system of claim 1, wherein the computer is operative to identify a plurality of most likely branch positions having VB views closest to the received RB view.

3. The system of claim 1, wherein the computer uses domain-specific information regarding general bronchoscope position within the airway tree.

4. The system of claim 3, wherein the domain-specific information includes current lung designation or lung lobar position.

5. The system of claim 1, wherein the computer is operative to identify an optimum branch position having a VB view closest to each of a plurality of RB views, and wherein the multiple RB views are associated with the same or multiple branch positions within an airway tree.

6. The system of claim 1, wherein the intra-branch search performed by the computer comprises a fast search over all the branches in a global airway-bifurcation search space using a weighted normalized sum of squares distance metric to determine a best match.

7. The system of claim 1, wherein the intra-branch search performed by the computer comprises uses pre-computed lumen region enclosing rectangles and a fast local registration refinement.

8. The system of claim 1, wherein the intra-branch search performed by the computer comprises is carried out along a centerline of each airway branch.

9. The system of claim 1, wherein the matched VB view and RB view are correlated to a bifurcation feature.

10. The system of claim 1, wherein the computer is further operative to match the VB view and the RB view along the airway branches including portions of the airway branches between bifurcations.

11. A global registration method useful in bronchoscopic guidance and other applications, the method comprising the steps of:

obtaining virtual bronchoscopy (VB) renderings of a 3D airway tree, the renderings including VB views and branch positions within the airway tree;

receiving a real bronchoscopic (RB) view at an unknown or arbitrary location along an unknown or arbitrary airway branch of the airway tree;

providing and executing an algorithm on a computer operative to use the RB view to search and analyze the VB renderings to identify the VB view and branch position that most closely matches the received RB view; and

outputting the 3D position of the bronchoscope within the airway tree based upon the VB view and branch position identified in the VB view;

wherein the search includes;

an intra-branch search to identify an optimal VB view along each airway branch of the plurality of airway branches of the airway tree that matches the received RB view, wherein the intra-branch search spans less than all possible view positions in each branch; and

an inter-branch search subsequent to the intra-branch search in which at least one of the optimal VB views from the intra-branch search that most closely matches the RB view is identified and used for determining the 3D position of the bronchoscope.

12. The method of claim 11, wherein the intra-branch search is carried out along a predefined set of view sites.

13. The method of claim 11, wherein the intra-branch search is carried out along a centerline of each airway branch.

14. The method of claim 11, wherein the matched VB view and RB view are correlated to a bifurcation feature.

15. The method of claim 11, wherein the matched VB view and the RB view are along the airway branches including portions of the airway branches between bifurcations.

16. The method of claim 11, wherein the intra-branch search includes

carrying out a fast scan through all predefined view sites along a centerline of the branch;

and

rendering view position adjustments using fast local registration by using a number of different roll angles of the bronchoscope at each of the predefined view sites.

17. The method of claim 11, wherein the outputting comprises displaying the 3D position of the bronchoscope within the airway tree based upon the VB view and branch position identified in the VB view.

18. The method of claim 11, wherein the inter-branch search is carried out using a dissimilarity metric.

19. The method of claim 11, wherein the inter-branch search is given by:

20. The method of claim 11, wherein the intra-branch search is given by: