System and method for guiding and tracking a region of interest using an endoscope

• US 10,292,570 B2
• Filed: 03/14/2017
• Issued: 05/21/2019
• Est. Priority Date: 03/14/2016
• Status: Active Grant

• Current Assignee: EndoChoice Incorporated (Boston Scientific Corporation)
• Original Assignee: EndoChoice Incorporated (Boston Scientific Corporation)
• Inventors: Kronman, Achia, Sharoni, Ron
• Primary Examiner(s): Kumar, Pankaj
• Assistant Examiner(s): Newlin, Timothy R
• Application Number:
  US15/458,239
• Publication Number:
  US 20170258295A1
• Time in Patent Office: 798 Days
• CPC Class Codes: A61B 1/000094 extracting biological structures
• CPC Class Codes: A61B 1/0004 for electronic operation
• CPC Class Codes: A61B 1/00042 for mechanical operation
• CPC Class Codes: A61B 1/0005 combining images e.g. side-by-side, superimposed or tiled
• CPC Class Codes: A61B 1/00114 Electrical cables in or with an endoscope
• CPC Class Codes: A61B 1/0016 using motor drive units
• CPC Class Codes: A61B 1/018 for receiving instruments
• CPC Class Codes: A61B 1/05 characterised by the image sensor, e.g. camera, being in the distal end portion
• CPC Class Codes: A61B 2034/2055 Optical tracking systems
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: H04N 23/555 for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes
• CPC Class Codes: H04N 23/56 provided with illuminating means

Abstract

A system and method of repositioning a region of interest within an endoscopic image from a first location to a target location on the display screen of an endoscopy system is provided. The endoscopy system includes an insertion portion having a tip section with at least one viewing element for capturing images of a body cavity having the region of interest. The method includes detecting a region of interest within a body cavity, displaying the region of interest on a display screen in communication with the endoscopy system, marking the region of interest on the display screen, selecting a target location on the display screen, estimating a transformation to be applied to the region of interest, and applying the estimated transformation for causing the region of interest to be displayed at the target location of the display screen.

Claims

1. A method of displaying a region of interest within an endoscopic image on a target area of a display screen of an endoscopy system comprising an insertion portion having a tip section with at least a front viewing element and a side viewing element for capturing images of a body cavity comprising the region of interest, the display screen being coupled with the front viewing element and the side viewing element, the method comprising:

capturing an image of the region of interest through one of the front viewing element or side viewing element;

displaying the region of interest on the display screen;

recording a demarcation of the region of interest on the display screen;

recording a demarcation of the target area on the display screen;

determining a transformation to be applied to the region of interest; and

applying the determined transformation to provide guidance to direct a movement of the insertion portion toward the target area based on said recorded demarcation of the target area such that the region of interest is positioned in the target area on the display.

2. The method of claim 1 wherein the target area comprises the center of the display screen.

3. The method of claim 1 wherein the region of interest is a polyp.

4. The method of claim 1 wherein the pre-defined display screen is a central display screen coupled with the front viewing element and wherein the insertion portion comprises a working channel located proximate to the front viewing element.

5. The method of claim 1 wherein a user marks the region of interest by using a graphical user interface (GUI) and selecting a portion of the pre-defined display using a mouse click, a touch screen, a hand gesture, or eye tracking.

6. The method of claim 1 wherein determining a transformation to be applied to the region of interest comprises determining an angle between the region of interest and the front viewing element.

7. The method of claim 1 wherein applying the determined transformation comprises generating and electronically displaying a visual guide that directs a movement of the insertion portion such that the region of interest moves into the target area on the display screen.

8. The method of claim 1 wherein applying the determined transformation comprises using motors in said endoscope to automatically move the insertion portion in order to position the region of interest in the target area.

9. An endoscope comprising an insertion portion having a tip section with at least one viewing element for capturing images of a body cavity comprising a region of interest, the at least one viewing element being in data communication with a display screen, the display screen being coupled with a processor for executing a plurality of programmatic instructions, stored in non-transient memory, for displaying the region of interest within an endoscopic image on a target area of the display screen, the plurality of programmatic instructions comprising:

capturing an image of the region of interest through the at least one viewing element;

displaying the region of interest on the display screen;

recording a demarcation of the region of interest on the display screen;

recording a demarcation of the target area on the display screen;

determining a transformation to be applied to the region of interest; and

applying the determined transformation to provide guidance to direct a movement of the insertion portion toward the target area based on said recorded demarcation of the target area such that the region of interest is positioned in the target area on the display screen.

10. The method of claim 9 wherein determining a transformation to be applied to the region of interest comprises determining an angle between the region of interest and the at least one viewing element.

11. The method of claim 9 wherein applying the determined transformation comprises generating and electronically displaying a visual guide that directs a movement of the insertion portion such that the region of interest moves into the target area on the display screen.

12. A method of repositioning a region of interest from a first location to a second location on at least one display screen of an endoscopy system, said endoscopy system having an insertion portion with at least one viewing element, comprising:

selecting a first location comprising a detected region of interest on the at least one display screen;

selecting a second location on the at least one display screen;

estimating a transformation to be applied to guide a movement of said region of interest toward said second location;

applying said estimated transformation;

evaluating if the region of interest is displayed at said second location on the at least one display screen; and

if said region of interest is not displayed at said second location, repeatedly estimating a new transformation based on a most recent location of the region of interest and guiding a movement of said region of interest toward said second location;

wherein applying the estimated transformation comprises generating and electronically displaying a visual guide that directs a movement of the insertion portion such that the region of interest moves into the second location on the display.

13. The method of claim 12, wherein said first location and said second location are on one display screen.

14. The method of claim 12, wherein said second location is substantially at a center of the at least one display screen.

15. The method of claim 12, wherein said second location is at a focal point of images being captured by the at least one viewing element.

16. The method of claim 12, wherein said selecting of the first location or the selecting of the second location are achieved by detecting a mouse click, a touch screen, a hand gesture, or eye movement applied to said at least one display screen.

17. The method of claim 12, wherein said estimation of transformation to be applied to the region of interest comprises determining an angle between the region of interest and the at least one viewing element.

18. The method of claim 12, wherein applying the estimated transformation comprises using motors in said endoscope to automatically move the insertion portion in order to position the region of interest in the second location.

19. A method of repositioning a region of interest from a first location to a second location on at least one display screen of an endoscopy system, said endoscopy system having an insertion portion with at least one viewing element, comprising:

selecting a first location comprising a detected region of interest on the at least one display screen;

selecting a second location on the at least one display screen;

estimating a transformation to be applied to guide a movement of said region of interest toward said second location;

applying said estimated transformation;

evaluating if the region of interest is displayed at said second location on the at least one display screen; and

if said region of interest is not displayed at said second location, repeatedly estimating a new transformation based on a most recent location of the region of interest and guiding a movement of said region of interest toward said second location;

wherein applying the estimated transformation comprises using motors in said endoscope to automatically move the insertion portion in order to position the region of interest in the second location.