Method and system for providing visual guidance to an operator for steering a tip of an endoscopic device toward one or more landmarks in a patient

• US 8,337,397 B2
• Filed: 03/26/2009
• Issued: 12/25/2012
• Est. Priority Date: 03/26/2009
• Status: Active Grant

• Current Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Original Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Inventors: Prisco, Giuseppe, Duindam, Vincent
• Primary Examiner(s): Leubecker, John P
• Assistant Examiner(s): COLQUE, RONALD DAVID
• Application Number:
  US12/411,515
• Publication Number:
  US 20100249507A1
• Time in Patent Office: 1,370 Days
• Field of Search: 600/117, 600/111, 600/103, 600/118, 600/109, 600/424, 600/173, 600/160, 600/105, 600/178, 345/427, 345/420, 345/424, 345/629, 345/630, 382/103, 434/262
• US Class Current: 600/117
• CPC Class Codes: A61B 1/000094 extracting biological structures
• CPC Class Codes: A61B 1/0002 provided with data storages
• CPC Class Codes: A61B 1/00045 Display arrangement
• CPC Class Codes: A61B 1/0005 combining images e.g. side-by-side, superimposed or tiled
• CPC Class Codes: A61B 1/00055 for alerting the user
• CPC Class Codes: A61B 1/00147 Holding or positioning arrangements
• CPC Class Codes: A61B 1/0016 using motor drive units
• CPC Class Codes: A61B 1/00165 with light-conductive means, e.g. fibre optics A61B1/07 takes precedence
• CPC Class Codes: A61B 1/00193 adapted for stereoscopic vision
• CPC Class Codes: A61B 1/00194 adapted for three-dimensional imaging
• CPC Class Codes: A61B 1/0051 with controlled bending of insertion part
• CPC Class Codes: A61B 1/009 with bending or curvature detection of the insertion part
• CPC Class Codes: A61B 1/01 Guiding arrangements therefore
• CPC Class Codes: A61B 1/05 characterised by the image sensor, e.g. camera, being in the distal end portion
• CPC Class Codes: A61B 1/273 for the upper alimentary canal, e.g. oesophagoscopes, gastroscopes
• CPC Class Codes: A61B 2034/2061 using shape-sensors, e.g. fiber shape sensors with Bragg gratings
• CPC Class Codes: A61B 5/066 Superposing sensor position on an image of the patient, e.g. obtained by ultrasound or x-ray imaging

Abstract

Landmark directional guidance is provided to an operator of an endoscopic device by displaying graphical representations of vectors adjacent a current image captured by an image capturing device disposed at a tip of the endoscopic device and being displayed at the time on a display screen, wherein the graphical representations of the vectors point in directions that the endoscope tip is to be steered in order to move towards associated landmarks such as anatomic structures in a patient.

Claims

1. A computer implemented method for providing guidance to steer an endoscopic device towards one or more landmarks in a patient, the method comprising:

determining a current position of a reference point on the endoscopic device relative to a reference frame;

determining a first vector from the current position of the reference point to a position of a first landmark relative to the reference frame;

transforming the first vector from the reference frame to an image reference frame that corresponds to the perspective of an image capturing device disposed at a tip of the endoscopic device; and

displaying a graphical representation of the transformed first vector along with and positioned relative to a current image captured by the image capturing device on a display screen so as to indicate a direction to steer the tip of the endoscopic device towards the first landmark.

2. The method according to claim 1, wherein one or more optical fibers configured with strain sensors extend through the endoscopic device to provide position and shape information of the endoscopic device.

3. The method according to claim 1, wherein the current image is a stereoscopic image, the display screen is a three-dimensional display screen, and the graphical representation of the transformed first vector is a three-dimensional arrow pointing towards the direction to steer the tip of the endoscopic device towards the first landmark.

4. The method according to claim 1, further comprising prior to determining the first vector:

receiving an indication to establish the first landmark;

determining a position of the reference point at the time of receiving the indication to establish the first landmark;

storing the position of the reference point at the time of receiving the indication to establish the first landmark in a memory device; and

referencing the stored position as the position of the first landmark.

5. The method according to claim 4, further comprising prior to determining the first vector:

capturing an image of the first landmark at a first opportunity after receiving the indication to establish the first landmark; and

storing the captured image of the first landmark in the memory device;

wherein the displaying of the graphical representation of the transformed first vector along with the current image on the display screen further includes displaying the captured image of the first landmark on the display screen proximate to the graphical representation of the transformed first vector.

6. The method according to claim 1, further comprising:

determining a second vector from the current position of the reference point to a position of a second landmark relative to the reference frame;

transforming the second vector from the reference frame to the image reference frame; and

displaying a graphical representation of the transformed second vector along with the current image on the display screen so as to indicate a direction to steer the tip of the endoscopic device towards the second landmark.

7. The method according to claim 1, further comprising:

periodically repeating the determination of the current reference point position, the determination of the first vector, the transformation of the first vector, and the display of the graphical representation of the transformed first vector along with the current image on the display screen until an indication to stop such periodically repeated actions is received.

8. The method according to claim 1, wherein a size of the graphical representation of the transformed first vector is related to the magnitude of the transformed first vector.

9. The method according to claim 1, wherein a color of the graphical representation of the transformed first vector is a color associated with the first landmark.

10. The method according to claim 1, wherein the displaying of the graphical representation of the transformed first vector comprises displaying the graphical representation adjacent a side of the displayed current image that is indicative of the direction to steer the tip of the endoscopic device towards the first landmark.

11. The method according to claim 1, further comprising:

storing positions of a plurality of landmarks including the position of the first landmark in a memory device prior to determining vectors from the current position of the reference point to the positions of the plurality of landmarks.

12. The method according to claim 11, wherein the positions of the plurality of landmarks are based upon preoperative measured data.

13. The method according to claim 1, further comprising:

displaying computer models of the patient and the first landmark on a second display screen.

14. The method according to claim 13, wherein a color of the graphical representation of the transformed first vector is a same color as the computer model of the first landmark being displayed on the second display screen.

15. The method according to claim 1, further comprising:

displaying computer models of the patient and the first landmark on the display screen.

16. The method according to claim 15, wherein a color of the graphical representation of the transformed first vector is a same color as the computer model of the first landmark being displayed on the display screen.