System and method for image-based alignment of an endoscope

• US 9,055,881 B2
• Filed: 05/01/2005
• Issued: 06/16/2015
• Est. Priority Date: 04/26/2004
• Status: Active Grant

• Current Assignee: Covidien LP (Medtronic Plc)
• Original Assignee: Super Dimension Ltd.
• Inventors: Gilboa, Pinhas, Blecher, Danny, Greenberg, Benny
• Primary Examiner(s): Nguyen, Anhtuan T
• Assistant Examiner(s): CHOU, WILLIAM B
• Application Number:
  US11/568,260
• Publication Number:
  US 20080262297A1
• Time in Patent Office: 3,698 Days
• Field of Search: 600/103, 600/109, 600/117, 600/188, 600/424, 600/921
• US Class Current: 1/1
• CPC Class Codes: A61B 1/0005 combining images e.g. side-by-side, superimposed or tiled
• CPC Class Codes: A61B 1/00128 mechanical, e.g. for tubes or pipes
• CPC Class Codes: A61B 1/00137 End pieces at either end of the endoscope, e.g. caps, seals or forceps plugs
• CPC Class Codes: A61B 1/0014 Fastening element for attaching accessories to the outside of an endoscope, e.g. clips, clamps or bands
• CPC Class Codes: A61B 1/00147 Holding or positioning arrangements
• CPC Class Codes: A61B 1/0052 Constructional details of control elements, e.g. handles
• CPC Class Codes: A61B 1/012 characterised by internal passages or accessories therefor
• CPC Class Codes: A61B 1/018 for receiving instruments
• CPC Class Codes: A61B 1/042 characterised by a proximal camera, e.g. a CCD camera
• CPC Class Codes: A61B 1/05 characterised by the image sensor, e.g. camera, being in the distal end portion
• CPC Class Codes: A61B 1/2676 Bronchoscopes
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 5/055 involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging
• CPC Class Codes: A61B 5/06 Devices, other than using radiation, for detecting or locating foreign bodies for removing same A61B17/50 ; determining position of probes within or on the body of the patient
• CPC Class Codes: A61B 5/062 using magnetic field
• CPC Class Codes: A61B 5/415 the glands, e.g. tonsils, adenoids or thymus
• CPC Class Codes: A61B 5/418 lymph vessels, ducts or nodes
• CPC Class Codes: A61B 6/032 Transmission computed tomography [CT]
• CPC Class Codes: A61B 6/037 Emission tomography
• CPC Class Codes: A61B 8/13 Tomography A61B8/10, A61B8/12 take precedence; tomography for radiation diagnosis A61B6/02
• CPC Class Codes: A61B 90/50 : Supports for surgical instruments, e.g. articulated arms
• CPC Class Codes: A61B 90/57 : Accessory clamps

Abstract

Systems and methods for endoscopic procedures employ a first technique to ensure initial correct alignment of an endoscope (100) with a desired target (10). A reference image (51) is then acquired from an imaging arrangement associated with the endoscope. During a subsequent stage of the procedure, tracking of the endoscope position relative to the target is performed partially or entirely by image-based tracking by comparing features in real-time video image (52) produced by imaging arrangement with features in the reference image (51). The feature comparison may be performed visually by a user, or may be automated to offer more specific corrective suggestions to the user.

Claims

1. A method for confirming correct alignment of a distal end of an endoscope including an imaging arrangement during an endoscopic procedure, the method comprising:

positioning the distal end of the endoscope adjacent to a target location and capturing a reference image using the imaging arrangement;

sensing a real-time video image using the imaging arrangement;

comparing features of the real-time video image with the reference image to determine whether the endoscope is correctly aligned;

if the endoscope is not correctly aligned;

measuring a mismatch between the reference image and the real-time video image;

providing a visual indication of the mismatch, the visual indication including directionality and magnitude information; and

adjusting a position of the endoscope based on the directionality and magnitude information derived from the mismatch; and

correlating a sub-window of the reference image provided on a display, and centered on the target location, with a corresponding sliding window of the real-time video image provided on the display.

2. The method of claim 1, wherein said step of positioning employs a target location identified in three-dimensional image data of a region of a body to be treated.

3. The method of claim 2, wherein said three-dimensional image data is derived from an imaging technique selected from:

computerized tomography;

magnetic resonance imaging;

positron emission tomography; and

ultrasound.

4. The method of claim 1, wherein the positioning step employs a position sensor associated with the distal end of the endoscope, said position sensor configured to aid in the measuring of the mismatch between the reference image and the real-time video image.

5. The method of claim 4, wherein the positioning step is performed by comparing a position of the distal end of the endoscope as measured in the measuring step and the target location.

6. The method of claim 4, wherein said position sensor is part of an elongated element deployed within a working channel of the endoscope, and wherein said elongated element is withdrawn from the working channel prior to said comparing step.

7. The method of claim 1, wherein said target location is not visible in said reference image.

8. The method of claim 1, further comprising displaying simultaneously said reference image and said real-time video image to facilitate the visual indication of the mismatch.

9. The method of claim 1, wherein said comparing step includes co-processing said reference image and at least one frame from the real-time video image, the method further comprising generating an alarm signal if the visual indication of the mismatch exceeds a predefined value.

10. The method of claim 1, wherein said comparing includes co-processing said reference image and at least one frame from said real-time video to determine a displacement correction required to compensate for erroneous movement of the endoscope, the method further comprising generating a display indicative to a user of said displacement correction required to compensate for the erroneous movement of the endoscope.

11. The method of claim 1, wherein said comparing includes co-processing said reference image and at least one frame from said real-time video to determine a transformation relating said real-time video frame to said reference image, the method further comprising generating a display corresponding to said real-time video with addition of an indication of a target location, position of said indication being derived at least in part by use of said transformation.

12. The method of claim 1, wherein the endoscope is a bronchoscope.

13. A system for ensuring correct alignment of an endoscope during performance of an endoscopic procedure, the system comprising:

an endoscope having a distal end for insertion into a body;

an imaging arrangement associated with said endoscope and configured to generate a real-time video image of a region beyond said distal end; and

a processing system associated with said imaging arrangement and configured to;

(i) in an initial state of alignment with a target location, derive from said imaging arrangement a reference image corresponding to correct alignment with the target location,

(ii) derive from said imaging arrangement real-time images of the region beyond said distal end,

(iii) co-process said reference image and said real-time images to determine a current alignment status of said endoscope with the target location,

(iv) if said endoscope is not currently aligned;

measure a mismatch between the reference image and the real-time video image;

provide a visual indication of the mismatch, the visual indication including directionality and magnitude information; and

adjust a position of the endoscope based on the directionality and magnitude information derived from the mismatch; and

(v) correlate a sub-window of the reference image provided on a display, and centered on the target location, with a corresponding sliding window of the real-time video image provided on the display.

14. The system of claim 13, wherein said processing system is configured to co-process said reference image and said real-time images, said processing system further generating an alarm signal if the visual indication of the mismatch exceeds a predefined value.

15. The system of claim 13, further comprising a display for displaying at least said real-time images to a user, wherein said processing system is configured to co-process said reference image and said real-time images to determine a displacement correction required to compensate for erroneous movement of the endoscope, said processing system further generating an indication on said display indicative to a user of said displacement correction required to compensate for the erroneous movement of the endoscope.

16. The system of claim 13, further comprising a display for displaying at least said real-time images to a user, wherein said processing system is configured to co-process said reference image and said real-time images to determine a transformation relating said real-time video frame to said reference image, said processing system further generating on said display an indication of a target location, position of said indication being derived at least in part by use of said transformation.

17. The system of claim 13, further comprising a position sensor carried by an elongated element removably deployable along a working channel of said endoscope.