System and method for image-based alignment of an endoscope

• US 10,321,803 B2
• Filed: 04/23/2015
• Issued: 06/18/2019
• Est. Priority Date: 04/26/2004
• Status: Active Grant

• Current Assignee: Covidien LP (Medtronic Plc)
• Original Assignee: Covidien LP (Medtronic Plc)
• Inventors: Gilboa, Pinhas, Blecher, Danny, Greenburg, Benny
• Primary Examiner(s): Leubecker, John P
• Assistant Examiner(s): Chou, William B
• Application Number:
  US14/694,181
• Publication Number:
  US 20150223668A1
• Time in Patent Office: 1,517 Days
• Field of Search: 600103, 600109, 600117, 600188, 600424, 600921
• CPC Class Codes: A61B 1/0005 combining images e.g. side-by-side, superimposed or tiled
• CPC Class Codes: A61B 1/00128 mechanical, e.g. for tubes or pipes
• CPC Class Codes: A61B 1/00137 End pieces at either end of the endoscope, e.g. caps, seals or forceps plugs
• CPC Class Codes: A61B 1/0014 Fastening element for attaching accessories to the outside of an endoscope, e.g. clips, clamps or bands
• CPC Class Codes: A61B 1/00147 Holding or positioning arrangements
• CPC Class Codes: A61B 1/0052 Constructional details of control elements, e.g. handles
• CPC Class Codes: A61B 1/012 characterised by internal passages or accessories therefor
• CPC Class Codes: A61B 1/018 for receiving instruments
• CPC Class Codes: A61B 1/042 characterised by a proximal camera, e.g. a CCD camera
• CPC Class Codes: A61B 1/05 characterised by the image sensor, e.g. camera, being in the distal end portion
• CPC Class Codes: A61B 1/2676 Bronchoscopes
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 5/055 involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging
• CPC Class Codes: A61B 5/06 Devices, other than using radiation, for detecting or locating foreign bodies for removing same A61B17/50 ; determining position of probes within or on the body of the patient
• CPC Class Codes: A61B 5/062 using magnetic field
• CPC Class Codes: A61B 5/415 the glands, e.g. tonsils, adenoids or thymus
• CPC Class Codes: A61B 5/418 lymph vessels, ducts or nodes
• CPC Class Codes: A61B 6/032 Transmission computed tomography [CT]
• CPC Class Codes: A61B 6/037 Emission tomography
• CPC Class Codes: A61B 8/13 Tomography A61B8/10, A61B8/12 take precedence; tomography for radiation diagnosis A61B6/02
• CPC Class Codes: A61B 90/50 : Supports for surgical instruments, e.g. articulated arms
• CPC Class Codes: A61B 90/57 : Accessory clamps

Abstract

Systems and methods for endoscopic procedures employ a first technique to ensure initial correct alignment of an endoscope (100) with a desired target (10). A reference image (51) is then acquired from an imaging arrangement associated with the endoscope. During a subsequent stage of the procedure, tracking of the endoscope position relative to the target is performed partially or entirely by image-based tracking by comparing features in real-time video image (52) produced by imaging arrangement with features in the reference image (51). The feature comparison may be performed visually by a user, or may be automated to offer more specific corrective suggestions to the user.

Claims

1. A method for confirming correct alignment of a distal end portion of an endoscope including an imaging arrangement, the method comprising:

positioning a distal end portion of an endoscope adjacent to a target location to capture a reference image using an imaging arrangement of the endoscope;

generating a real-time video image;

comparing features of the real-time video image with the reference image to determine whether the endoscope is correctly aligned;

measuring, if the endoscope is not correctly aligned, a mismatch between the reference image and the real-time video image;

correlating a sub-window of at least a portion of the reference image provided on a display, the sub-window centered on the target location, with a corresponding sliding window of the real-time video image provided on the display;

displaying the sub-window superimposed over the sliding window;

designating a marker corresponding to a center of the target location; and

updating a location of the marker in the sliding window when a view of the real-time video changes.

2. The method of claim 1, further comprising providing a visual indication of the mismatch, the visual indication including directionality and magnitude information.

3. The method of claim 2, further comprising adjusting a position of the endoscope based on the directionality and magnitude information derived from the mismatch.

4. The method of claim 1, further comprising rotating the sub-window.

5. The method of claim 4, further comprising tracking the target location during rotation of the sub-window.

6. The method of claim 5, further comprising employing planar transformations to approximate distortions caused by different viewing angles of the target location.

7. The method of claim 1, further comprising employing the target location in three-dimensional image data of a region of a body to be treated.

8. The method of claim 7, wherein the three-dimensional image data is derived from an imaging technique selected from:

computerized tomography, magnetic resonance imaging, positron emission tomography, and ultrasound.

9. A system for confirming correct alignment of an endoscope, the system comprising:

an endoscope having a distal end portion for insertion into a body;

an imaging arrangement configured to generate a real-time video image of a region beyond the distal end portion; and

a processing system associated with the imaging arrangement and configured to;

(i) derive, when the imaging arrangement is in an initial state of alignment with a target location, a reference image,

(ii) derive from the imaging arrangement real-time images of the region beyond the distal end portion,

(iii) co-process the reference image and the real-time images to determine a current alignment status of the endoscope with the target location, wherein a mismatch is measured, if the endoscope is not correctly aligned, between the reference image and the real-time video images,

(iv) correlate a sub-window of the reference image provided on a display, the sub-window centered on the target location, with a corresponding sliding window of the real-time video image provided on the display;

(v) designate a marker corresponding to a center of the target location; and

(vi) update a location of the marker in the sliding window when a view of the real-time video changes.

10. The system of claim 9, wherein a visual indication of the mismatch is provided, the visual indication including directionality and magnitude information.

11. The system of claim 10, wherein a position of the endoscope is adjusted based on the directionality and magnitude information derived from the mismatch.

12. The system of claim 9, wherein the sub-window is rotated.

13. The system of claim 12, wherein the target location is tracked during rotation of the sub-window.

14. The system of claim 13, wherein planar transformations are employed to approximate distortions caused by different viewing angles of the target location.

15. The system of claim 9, wherein the target location is employed in three-dimensional image data of the region of the body to be treated.

16. The system of claim 15, wherein the three-dimensional image data is derived from an imaging technique selected from:

computerized tomography, magnetic resonance imaging, positron emission tomography, and ultrasound.

17. A method for confirming correct alignment of a distal end portion of an endoscope including an imaging arrangement, the method comprising:

positioning a distal end portion of an endoscope adjacent to a target location to capture a reference image using an imaging arrangement of the endoscope;

generating a real-time video image;

comparing features of the real-time video image with the reference image to determine whether the endoscope is correctly aligned;

correlating a sub-window of at least a portion of the reference image provided on a display, the sub-window centered on the target location, with a corresponding sliding window of the real-time video image provided on the display;

displaying the sub-window superimposed over the sliding window;

rotating the sub-window;

designating a marker corresponding to a center of the target location; and

updating a location of the marker in the sliding window when a view of the real-time video changes.

18. A system for confirming correct alignment of an endoscope, the system comprising:

an endoscope having a distal end portion for insertion into a body;

an imaging arrangement configured to generate a real-time video image of a region beyond the distal end portion; and

a processing system associated with the imaging arrangement and configured to;

(i) derive, when the imaging arrangement is in an initial state of alignment with a target location, a reference image,

(ii) derive from the imaging arrangement real-time images of the region beyond the distal end portion,

(iii) co-process the reference image and the real-time images to determine a current alignment status of the endoscope with the target location,

(iv) correlate a sub-window of the reference image provided on a display, the sub-window centered on the target location, with a corresponding sliding window of the real-time video image provided on the display, wherein the sub-window is rotated;

(v) designate a marker corresponding to a center of the target location; and

(vi) update a location of the marker in the sliding window when a view of the real-time video changes.