Hybrid registration method

• US 10,674,936 B2
• Filed: 09/04/2019
• Issued: 06/09/2020
• Est. Priority Date: 06/06/2008
• Status: Active Grant

• Current Assignee: Covidien LP (Medtronic Plc)
• Original Assignee: Covidien LP (Medtronic Plc)
• Inventors: Averbuch, Dorian, Zur, Oded, Weingarten, Oren P.
• Primary Examiner(s): Nguyen, Tu T
• Application Number:
  US16/560,863
• Publication Number:
  US 20190388005A1
• Time in Patent Office: 279 Days
• Field of Search: 382131
• CPC Class Codes: A61B 1/2676 Bronchoscopes
• CPC Class Codes: A61B 2017/00699 correcting for movement caused by respiration, e.g. by triggering
• CPC Class Codes: A61B 2034/2072 Reference field transducer attached to an instrument or patient
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 2090/3762 using computed tomography systems [CT]
• CPC Class Codes: A61B 2576/00 Medical imaging apparatus involving image processing or analysis A61B1/00009, A61B6/52 and A61B8/52 take precedence; image processing per se G06T; details of algorithms for analysing medical images G06T7/0012
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 5/06 Devices, other than using radiation, for detecting or locating foreign bodies for removing same A61B17/50 ; determining position of probes within or on the body of the patient
• CPC Class Codes: A61B 5/064 using markers A61B5/062 takes precedence
• CPC Class Codes: A61B 5/066 Superposing sensor position on an image of the patient, e.g. obtained by ultrasound or x-ray imaging
• CPC Class Codes: A61B 5/7246 using correlation, e.g. template matching or determination of similarity
• CPC Class Codes: A61B 90/361 Image-producing devices, e.g. surgical cameras
• CPC Class Codes: G06T 2207/10068 Endoscopic image
• CPC Class Codes: G06T 2207/10081 Computed x-ray tomography [CT]
• CPC Class Codes: G06T 2207/30061 Lung
• CPC Class Codes: G06T 7/0016 involving temporal comparison
• CPC Class Codes: G06T 7/33 using feature-based methods

Abstract

A registration method whereby a sensor-based approach is used to establish initial registration and whereby upon the commencement of navigating an endoscope, image-based registration methods are used in order to more accurately maintain the registration between the endoscope location and previously-acquired images. A six-degree-of-freedom location sensor is placed on the probe in order to reduce the number of previously-acquired images that must be compared to a real-time image obtained from the endoscope.

Claims

1. A method of registering an airway to a three-dimensional 3D model of the airway:

acquiring a plurality of computed tomography (CT) images of the airways of a patient;

generating a 3D model of the airways derived from the plurality of CT images;

generating a plurality of virtual endoscopic images derived from the 3D model;

acquiring an endoscopic image;

comparing the endoscopic image to the plurality of virtual endoscopic images to identify a match between the endoscopic image and a virtual endoscopic image of the plurality of virtual endoscopic images, wherein a viewpoint of the matching virtual endoscopic image substantially corresponds to a viewpoint of the endoscopic image;

determining a position associated with an endoscope in an airway of a patient to form a determined position;

comparing the determined position with a location in the 3D model corresponding to the matching virtual endoscopic image; and

when the determined position and the location in the 3D model substantially correspond, depicting a location of the endoscope in the 3D model.

2. The method of claim 1, wherein the determining the position includes recording a shape of a segment of the airway and identifying a segment in the 3D model that corresponds to the shape of the recorded segment.

3. The method of claim 1, wherein the determining the position includes determining a position of a sensor, and

wherein the sensor is an electromagnetic sensor, and the determining the position of the sensor includes detecting electromagnetic fields.

4. The method of claim 3, wherein the matching virtual endoscopic image defines a subgroup of the plurality of virtual endoscopic images corresponding to a vicinity of the sensor, and

wherein the plurality of virtual endoscopic images and the subgroup are stored in a database.

5. The method of claim 4, further comprising selecting an image from the subgroup that most accurately corresponds to an approximate location to update an initial registration using an image-based registration technique when the endoscope and the associated sensor are moved.

6. A method of registering sensor location data to images of a patient'"'"'s at least one body part, comprising:

generating location data corresponding to a location of a sensor within a patient'"'"'s body by recording a shape of a segment of the patient'"'"'s at least one body part and identifying a segment in a 3D model, which is derived from computed tomography (CT) images, that corresponds to the shape of the recorded segment;

comparing a plurality of previously acquired images of the patient'"'"'s at least one body part to a plurality of virtual images derived from the CT images to identify a subgroup of the plurality of previously acquired images of the patient'"'"'s at least one body part;

registering the location of the sensor relative to the subgroup of the plurality of previously acquired images of the patient'"'"'s at least one body part based on the location data; and

updating the registered location of the sensor based on one or more images selected from the subgroup.

7. The method of claim 6, wherein recording the shape of the segment of the patient'"'"'s at least one body part includes recording a plurality of locations of the sensor.

8. The method of claim 6, wherein recording the shape of the segment of the patient'"'"'s at least one body part includes recording a plurality of locations of a respective plurality of sensors disposed along the patient'"'"'s at least one body part.

9. The method of claim 6, wherein the sensor is an electromagnetic sensor and generating location data includes sensing electromagnetic fields.

10. The method of claim 6, wherein the plurality of previously acquired images are a plurality of virtual endoscopic images derived from computer tomography (CT) images stored in a memory, and a matching virtual endoscopic image defines a subset of the plurality of virtual endoscopic images corresponding to a vicinity of the sensor.

11. The method of claim 10, wherein the subset is a single virtual endoscopic image.

12. The method of claim 10, wherein the subset includes two or more virtual endoscopic images.

13. The method of claim 6, wherein the patient'"'"'s at least one body part is an airway of a lung.

14. A method of registering sensor location data to images of at least one body part, comprising:

generating location data corresponding to a location of a sensor within a patient'"'"'s body;

comparing a plurality of previously acquired images of the patient'"'"'s at least one body part to a plurality of virtual images derived from computed tomography (CT) images to identify a subgroup of the plurality of previously acquired images of the patient'"'"'s at least one body part;

registering the location of the sensor relative to the subgroup of the plurality of previously acquired images of the patient'"'"'s at least one body part based on the location data;

updating the registered location of the sensor based on one or more images selected from the subgroup; and

displaying the one or more images selected from the subgroup.

15. The method of claim 14, wherein the registered location is updated based on a single image and the single image is displayed.

16. The method of claim 14, wherein generating location data includes recording a shape of a segment of an airway and identifying a segment in a 3D model of lungs that corresponds to the shape of the recorded segment.

17. The method of claim 16, further comprising converting a plurality of computed tomography (CT) images of the lungs into a series of digitally-identified branches to develop the 3D model of the lungs.

18. The method of claim 14, wherein the sensor is an electromagnetic sensor and generating location data includes sensing electromagnetic fields.

19. The method of claim 14, wherein the plurality of previously acquired images are a plurality of virtual images derived from computer tomography (CT) images stored in a memory, and a virtual image matching an image acquired by an endoscope associated with the sensor defines a subset of the plurality of virtual images corresponding to a vicinity of the sensor.