METHOD AND SYSTEM FOR PROVIDING VISUAL GUIDANCE TO AN OPERATOR FOR STEERING A TIP OF AN ENDOSCOPIC DEVICE TOWARD ONE OR MORE LANDMARKS IN A PATIENT

• US 20100249507A1
• Filed: 03/26/2009
• Published: 09/30/2010
• Est. Priority Date: 03/26/2009
• Status: Active Grant

• Current Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Original Assignee: Intuitive Surgical Incorporated (Intuitive Surgical Holdings, LLC)
• Inventors: Prisco, Giuseppe, Duindam, Vincent
• Granted Patent: US 8,337,397 B2
• Time in Patent Office: Days
• US Class Current: 600/117
• CPC Class Codes: A61B 1/000094 extracting biological structures
• CPC Class Codes: A61B 1/0002 provided with data storages
• CPC Class Codes: A61B 1/00045 Display arrangement
• CPC Class Codes: A61B 1/0005 combining images e.g. side-by-side, superimposed or tiled
• CPC Class Codes: A61B 1/00055 for alerting the user
• CPC Class Codes: A61B 1/00147 Holding or positioning arrangements
• CPC Class Codes: A61B 1/0016 using motor drive units
• CPC Class Codes: A61B 1/00165 with light-conductive means, e.g. fibre optics A61B1/07 takes precedence
• CPC Class Codes: A61B 1/00193 adapted for stereoscopic vision
• CPC Class Codes: A61B 1/00194 adapted for three-dimensional imaging
• CPC Class Codes: A61B 1/0051 with controlled bending of insertion part
• CPC Class Codes: A61B 1/009 with bending or curvature detection of the insertion part
• CPC Class Codes: A61B 1/01 Guiding arrangements therefore
• CPC Class Codes: A61B 1/05 characterised by the image sensor, e.g. camera, being in the distal end portion
• CPC Class Codes: A61B 1/273 for the upper alimentary canal, e.g. oesophagoscopes, gastroscopes
• CPC Class Codes: A61B 2034/2061 using shape-sensors, e.g. fiber shape sensors with Bragg gratings
• CPC Class Codes: A61B 5/066 Superposing sensor position on an image of the patient, e.g. obtained by ultrasound or x-ray imaging

Abstract

Landmark directional guidance is provided to an operator of an endoscopic device by displaying graphical representations of vectors adjacent a current image captured by an image capturing device disposed at a tip of the endoscopic device and being displayed at the time on a display screen, wherein the graphical representations of the vectors point in directions that the endoscope tip is to be steered in order to move towards associated landmarks such as anatomic structures in a patient.

Claims

1. A computer implemented method for providing guidance to steer an endoscopic device towards one or more landmarks in a patient, the method comprising:

determining a current position of a reference point on the endoscopic device relative to a reference frame;

determining a first vector from the current position of the reference point to a position of a first landmark relative to the reference frame;

transforming the first vector from the reference frame to an image reference frame that corresponds to the perspective of an image capturing device disposed at a tip of the endoscopic device; and

displaying a graphical representation of the transformed first vector along with a current image captured by the image capturing device on a display screen so as to indicate a direction to steer the tip of the endoscopic device towards the first landmark.

2. The method according to claim 1, wherein one or more optical fibers configured with strain sensors extend through the endoscopic device to provide position and shape information of the endoscopic device.

3. The method according to claim 1, wherein the current image is a stereoscopic image, the display screen is a three-dimensional display screen, and the graphical representation of the transformed first vector is a three-dimensional arrow pointing towards the direction to steer the tip of the endoscopic device towards the first landmark.

4. The method according to claim 1, further comprising prior to determining the first vector:

receiving an indication to establish the first landmark;

determining a position of the reference point at the time of receiving the indication to establish the first landmark;

storing the position of the reference point at the time of receiving the indication to establish the first landmark in a memory device; and

referencing the stored position as the position of the first landmark.

5. The method according to claim 4, further comprising prior to determining the first vector:

capturing an image of the first landmark at a first opportunity after receiving the indication to establish the first landmark; and

storing the captured image of the first landmark in the memory device;

wherein the displaying of the graphical representation of the transformed first vector along with the current image on the display screen further includes displaying the captured image of the first landmark on the display screen proximate to the graphical representation of the transformed first vector.

6. The method according to claim 1, further comprising:

determining a second vector from the current position of the reference point to a position of a second landmark relative to the reference frame;

transforming the second vector from the reference frame to the image reference frame; and

displaying a graphical representation of the transformed second vector along with the current image on the display screen so as to indicate a direction to steer the tip of the endoscopic device towards the second landmark.

7. The method according to claim 1, further comprising:

periodically repeating the determination of the current reference point position, the determination of the first vector, the transformation of the first vector, and the display of the graphical representation of the transformed first vector along with the current image on the display screen until an indication to stop such periodically repeated actions is received.

8. The method according to claim 1, wherein a size of the graphical representation of the transformed first vector is related to the magnitude of the transformed first vector.

9. The method according to claim 1, wherein a color of the graphical representation of the transformed first vector is a color associated with the first landmark.

10. The method according to claim 1, wherein the displaying of the graphical representation of the transformed first vector comprises displaying the graphical representation adjacent a side of the displayed current image that is indicative of the direction to steer the tip of the endoscopic device towards the first landmark.

11. The method according to claim 1, further comprising:

storing positions of a plurality of landmarks including the position of the first landmark in a memory device prior to determining vectors from the current position of the reference point to the positions of the plurality of landmarks.

12. The method according to claim 11, wherein the positions of the plurality of landmarks are based upon preoperative measured data.

13. The method according to claim 1, further comprising:

displaying computer models of the patient and the first landmark on a second display screen.

14. The method according to claim 13, wherein a color of the graphical representation of the transformed first vector is a same color as the computer model of the first landmark being displayed on the second display screen.

15. The method according to claim 1, further comprising:

displaying computer models of the patient and the first landmark on the display screen.

16. The method according to claim 15, wherein a color of the graphical representation of the transformed first vector is a same color as the computer model of the first landmark being displayed on the display screen.

17. A system comprising:

a display screen;

an endoscopic device; and

one or more processors adapted to determine a current position of a reference point on the endoscopic device relative to a reference frame, determine a first vector from the current position of the reference point to a position of a first landmark relative to the reference frame, transform the first vector from the reference frame to an image reference frame that corresponds to the perspective of an image capturing device disposed at a tip of the endoscopic device, and display a graphical representation of the transformed first vector along with a current image captured by the image capturing device on a display screen so as to indicate a direction to steer the tip of the endoscopic device towards the first landmark.

18. The system according to claim 17, wherein the one or more processors includes a position processor, the system further comprising:

one or more optical fibers configured with strain sensors, wherein the one or more optical fibers extend through the endoscopic device to provide position and shape information of the endoscopic device to the position processor.

19. The system according to claim 17, wherein the current image is a stereoscopic image, the display screen is a three-dimensional display screen, and the graphical representation of the transformed first vector is a three-dimensional arrow pointing towards the direction to steer the tip of the endoscopic device towards the first landmark.

20. The system according to claim 17, wherein the one or more processors includes a display processor, the system further comprising:

a memory device;

wherein the display processor is adapted to receive an indication to establish the first landmark, determine a position of the reference point on the endoscopic device within a process cycle of receiving the indication to establish the first landmark, store the determined position of the reference point in the memory device, and reference the stored position as the position of the first landmark.

21. The system according to claim 20, wherein the display processor is adapted to store an image captured by the image capturing device of the first landmark in the memory device within a process cycle of receiving the indication to establish the first landmark, and subsequently retrieve and display the stored captured image of the first landmark proximate to the graphical representation of the transformed first vector when the transformed first vector is being displayed along with the current image on the display screen.

22. The system according to claim 17, wherein the processor is adapted to determine a second vector from the current position of the reference point on the endoscopic device to a position of a second landmark relative to the reference frame, transform the second vector from the reference frame to the image reference frame, and display a graphical representation of the transformed second vector along with the current image on the display screen so as to indicate a direction to steer the tip of the endoscopic device towards the second landmark.

23. The system according to claim 17, wherein the one or more processors is adapted to perform in each of a sequence of process cycles the determining of the current reference point position, the determining of the first vector, the transforming of the first vector, and the displaying of the graphical representation of the transformed first vector along with the current image on the display screen.

24. The system according to claim 17, wherein the one or more processors is adapted to display the graphical representation of the transformed first vector so that a size of the graphical representation is related to the magnitude of the transformed first vector.

25. The system according to claim 17, wherein the one or more processors is adapted to display the graphical representation of the transformed first vector so that a color of the graphical representation is a color associated with the first landmark.

26. The system according to claim 17, wherein the one or more processors is adapted to display the graphical representation of the transformed first vector adjacent a side of the displayed current image that is indicative of the direction to steer the tip of the endoscopic device towards the first landmark.

27. The system according to claim 17, further comprising:

a memory device;

wherein the one or more processors is adapted to store positions of a plurality of landmarks including the position of the first landmark in the memory device prior to determining vectors from the current position of the reference point to the positions of the plurality of landmarks.

28. The system according to claim 27, wherein the positions of the plurality of landmarks are based upon preoperative measured data.

29. The system according to claim 17, wherein the one or more processors is adapted to display computer models of a patient and the first landmark on a second display screen.

30. The system according to claim 29, wherein the one or more processors is adapted to display the graphical representation of the first vector on the display screen so that a color of the graphical representation of the transformed first vector is a same color as the computer model of the first landmark being displayed on the second display screen.

31. The system according to claim 17, wherein the one or more processors is adapted to display computer models of a patient and the first landmark on the display screen.

32. The system according to claim 31, wherein the one or more processors is adapted to display the graphical representation of the first vector on the display screen so that a color of the graphical representation of the transformed first vector is a same color as the computer model of the first landmark being displayed on the display screen.