Systems and methods for registration of multiple vision systems

• US 10,278,615 B2
• Filed: 08/12/2013
• Issued: 05/07/2019
• Est. Priority Date: 08/14/2012
• Status: Active Grant

• Current Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Original Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Inventors: Zhao, Tao, Chopra, Prashant
• Primary Examiner(s): Brutus, Joel F
• Application Number:
  US13/964,704
• Publication Number:
  US 20140051986A1
• Time in Patent Office: 2,094 Days
• Field of Search: 600424
• CPC Class Codes: A61B 1/000094 extracting biological structures
• CPC Class Codes: A61B 1/009 with bending or curvature detection of the insertion part
• CPC Class Codes: A61B 2034/2065 Tracking using image or pattern recognition
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 5/062 using magnetic field
• CPC Class Codes: A61B 5/066 Superposing sensor position on an image of the patient, e.g. obtained by ultrasound or x-ray imaging
• CPC Class Codes: G06T 2207/10068 Endoscopic image
• CPC Class Codes: G06T 2207/30021 Catheter; Guide wire
• CPC Class Codes: G06T 7/33 using feature-based methods

Abstract

A method comprises generating a model of an anatomic region and receiving a true image from an endoscopic image capture probe positioned within the anatomic region. The method further comprises identifying a true fiducial region in the true image and identifying a plurality of virtual tissue structures in the model of the anatomic region. The method further comprises matching one of the plurality of the virtual tissue structures with the true fiducial region and determining a probe pose of the endoscopic image capture probe from the matched one of the plurality of virtual tissue structures.

Claims

1. A method for using a flexible catheter in a medical procedure, the method being implemented using a processor and comprising:

receiving an endoscopic image from a distal end of an endoscopic image capture probe positioned within an anatomic region;

identifying a fiducial region in the endoscopic image;

defining a subregion of a model of the anatomic region, the subregion corresponding to an area surrounding a measured position of the endoscopic image capture probe, the subregion including a first portion situated distally from the distal end of the endoscopic image capture probe and a second portion situated proximally from the distal end of the endoscopic image capture probe, the measured position of the endoscopic image capture probe being received from a sensor system included in the endoscopic image capture probe;

identifying a search space defined by the subregion of the model, the search space including a plurality of virtual tissue structures from the subregion of the model;

matching the fiducial region in the endoscopic image with a matched one of the plurality of virtual tissue structures in the subregion of the model of the anatomic region, wherein the matched one of the plurality of virtual tissue structures is associated with a virtual probe position;

localizing a tip of the endoscopic image capture probe relative to the model of the anatomic region wherein localizing the tip includes determining an offset between the measured position of the endoscopic image capture probe and the virtual probe position; and

guiding movement of the endoscopic image capture probe in the anatomic region based on the offset.

2. The method of claim 1 further comprising determining a virtual probe orientation of the tip of the endoscopic image capture probe from the matched one of the plurality of virtual tissue structures.

3. The method of claim 1 further comprising displaying the matched one of the plurality of virtual tissue structures from the model of the anatomic region.

4. The method of claim 1, wherein the subregion surrounding the measured position of the endoscopic image capture probe is defined by at least one of a sensor tolerance or an expected anatomic motion about the measured position of the endoscopic image capture probe relative to the anatomic region.

5. The method of claim 1, wherein matching the fiducial region further comprises:

identifying one or more structural aspects of the fiducial region; and

identifying a subset of the plurality of virtual tissue structures,

wherein the subset of the plurality of virtual tissue structures includes the one or more structural aspects of the fiducial region.

6. The method of claim 5, wherein the fiducial region includes one or more lumens, and the one or more structural aspects includes at least one of lumen location, lumen orientation, lumen dimension, lumen contour, lumen anatomical label, and lumen shape.

7. The method of claim 5, wherein matching the fiducial region further comprises:

calculating an optimum similarity measure for each of the plurality of virtual tissue structures in the subset; and

selecting the matched one of the plurality of virtual tissue structures by identifying one of the plurality of virtual tissue structures in the subset exhibiting the optimum similarity measure.

8. The method of claim 7, wherein calculating the optimum similarity measure comprises:

iteratively rendering a plurality of views for each of the plurality of virtual tissue structures in the subset of the plurality of virtual tissue structures;

generating a plurality of interim similarity measures; and

selecting the optimum similarity measure from the plurality of interim similarity measures for each of the plurality of virtual tissue structures in the subset, the optimum similarity measure indicating a match to the fiducial region for the plurality of views.

9. The method of claim 1, further comprising registering the model of the anatomic region with the endoscopic image based on the localization of the tip of the endoscopic image capture probe.

10. The method of claim 1 further comprising:

removing the endoscopic image capture probe from a catheter body positioned within the anatomic region; and

inserting a tracked interventional instrument through the catheter body to conduct a procedure.

11. The method of claim 1, wherein the anatomic region comprises a lung.

12. A medical system comprising:

a flexible catheter sized to receive an image capture probe wherein the image capture probe is sized to extend within an anatomic region;

a sensor system; and

a processor configured to;

generate a model of the anatomic region, the model being three-dimensional;

receive an image from the image capture probe positioned within the anatomic region;

identify a fiducial region in the image received from the image capture probe;

define a subregion of the model corresponding to a three-dimensional area surrounding a measured position of the image capture probe, the measured position being received from the sensor system;

identify a search space defined by the subregion of the model to identify a plurality of virtual tissue structures in the subregion of the model of the anatomic region;

match a matched one of the plurality of the virtual tissue structures in the subregion of the model with the identified fiducial region in the image received from the image capture probe, wherein the matched one of the plurality of the virtual tissue structures is associated with a virtual probe position;

localize a tip of the image capture probe relative to the model of the anatomic region including determining an offset between the measured position of the image capture probe and the virtual probe position; and

guide movement of the image capture probe in the anatomic region based on the offset.

13. The medical system of claim 12 wherein the processor is further configured to determine a virtual probe orientation of the tip of the image capture probe from the matched one of the plurality of virtual tissue structures.

14. The medical system of claim 12, wherein the processor is further configured to generate a display of the matched one of the plurality of virtual tissue structures from the model of the anatomic region.

15. The medical system of claim 12, wherein the processor is further configured to

identify structural aspects of the fiducial region;

and

identify a subset of the plurality of virtual tissue structures matching the structural aspects of the fiducial region.

16. The medical system of claim 15, wherein the processor is further configured to

perform an iterative rendering of each of the plurality of virtual tissue structures in the subset to match the fiducial region;

and

identify a matched one of the plurality of virtual tissue structures based on the iterative rendering.

17. The medical system of claim 12, wherein the processor is further configured to register the model of the anatomic region to the image based on the localizing of the tip of the image capture probe.

18. The medical system of claim 17, further comprising a display system to display a rendering of the matched one of the plurality of virtual tissue structures based on the localizing of the tip of the image capture probe.

19. The medical system of claim 12, further comprising:

a steering system for steering a distal region of the flexible catheter, wherein the flexible catheter controls a probe pose; and

a control system for maintaining a catheter pose of the flexible catheter when the image capture probe is removed.

20. The medical system of claim 12, further comprising:

a flexible catheter sized to receive the image capture probe;

a steering system for steering a distal region of the flexible catheter, wherein the flexible catheter controls a probe pose; and

a tracked tool deployed through the flexible catheter, wherein the tracked tool is controlled by reference to a displayed virtual visualization image.

21. A method for using a flexible catheter in a medical procedure, the method comprising:

receiving a model of an anatomic region within a patient;

receiving an endoscopic image from an endoscopic image capture probe positioned within the anatomic region;

identifying a bifurcation structure in the endoscopic image;

defining a model subregion contained within the model of the anatomic region, the model subregion corresponding to an area surrounding a measured position of the endoscopic image capture probe received from a sensor system included in the endoscopic image capture probe;

identifying a search space defined by the model subregion of the model, the search space including a plurality of virtual bifurcation structures from the model subregion of the model;

matching the bifurcation structure in the endoscopic image with a matched one of the plurality of virtual bifurcation structures in the model subregion, wherein the matched one of the plurality of virtual bifurcation structures is associated with a virtual probe position;

determining an offset between the measured position of the endoscopic image capture probe and the virtual probe position; and

guiding movement of the endoscopic image capture probe in the anatomic region based on the offset.