Geometrically appropriate tool selection assistance for determined work site dimensions

• US 9,375,844 B2
• Filed: 03/14/2014
• Issued: 06/28/2016
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Original Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Inventors: Itkowitz, Brandon D., DiMaio, Simon P., Gerbi, Craig R., Rogers, Theodore W., Zhao, Wenyi
• Primary Examiner(s): Abdi, Amara
• Application Number:
  US14/210,881
• Publication Number:
  US 20140277736A1
• Time in Patent Office: 837 Days
• US Class Current: 1/1
• CPC Class Codes: B25J 9/1697 Vision controlled systems

Abstract

A robotic system includes a processor that is programmed to determine and cause work site measurements for user specified points in the work site to be graphically displayed in order to provide geometrically appropriate tool selection assistance to the user. The processor is also programmed to determine an optimal one of a plurality of tools of varying geometries for use at the work site and to cause graphical representations of at least the optimal tool to be displayed along with the work site measurements.

Claims

1. An apparatus comprising:

a display;

an image capture device disposed to capture images of a work site; and

a processor programmed to;

determine positions of one or more user specified points in the work site relative to an image capturing perspective of the image capture device;

translate the determined positions of the one or more user specified points so as to be relative to a target reference frame which is noncoincident with the image capturing perspective of the image capture device;

cause information of the translated positions of the one or more user specified points to be graphically displayed on the display so as to provide work site measurements relative to the target reference frame; and

display information of a plurality of tools of varying geometries along with the information of the translated positions of the one or more user specified points on the display, wherein the work site is accessible by each of the plurality of tools by the tool being inserted through an entry port located at an origin of the target reference frame.

2. The apparatus of claim 1, wherein the processor is programmed to:

determine an optimal one of the plurality of tools for use at the work site according to the geometries of the plurality of tools and distances to the translated positions of the one or more user specified points relative to the target reference frame; and

cause an indication of the optimal one of the plurality of tools to be displayed on the display.

3. The apparatus of claim 1, wherein the work site comprises one or more objects, and wherein the one or more user specified points are disposed on a surface topology of the one or more objects from the image capturing perspective of the image capture device.

4. The apparatus of claim 3, further comprising:

a retractor that is retracting at least a portion of one of the one or more objects.

5. The apparatus of claim 3, wherein the work site comprises a cavity, wherein the one or more objects are disposed within the cavity, and further comprising:

an insufflator for insufflating the cavity with a gas.

6. The apparatus of claim 1, wherein the processor is programmed to determine the positions of the one or more user specified points in the work site relative to the image capture device by determining a depth for each of the one or more user specified points relative to the image capture device by analyzing information of light reflected from the one or more objects.

7. The apparatus of claim 1, wherein the image capture device captures a pair of stereo images of the work site, and wherein the processor is programmed to determine the positions of one or more user specified points in a work site relative to the image capture device by determining a depth for each of the one or more user specified points relative to the image capture device by analyzing the stereo images captured by the image capture device.

8. The apparatus of claim 1, wherein the image capture device is a stereo camera, and wherein the processor is programmed to receive information of the one or more user specified points in the work site by receiving indications of user specified points on one or more objects at the work site as the image capturing perspective of the stereo camera is directed towards different locations on the one or more objects.

9. The apparatus of claim 1, further comprising:

a graphical user interface for receiving one or more points indicated on a screen of the display while the screen of the display is displaying images captured by the image capture device of the work site;

wherein the processor is programmed to determine positions of the one or more user specified points in the work site by translating the one or more points indicated on the screen of the display to the image capturing perspective of the image capture device.

10. The apparatus of claim 1, further comprising:

a telestrator for receiving one or more points indicated on a screen of the display while the screen of the display is displaying images captured by the image capture device of the work site;

wherein the processor is programmed to determine positions of the one or more user specified points in the work site by translating the one or more points indicated on the screen of the display to the image capturing perspective of the image capture device.

11. The apparatus of claim 1, further comprising:

a gaze tracker for receiving one or more points indicated on a screen of the display while the screen of the display is displaying images captured by the image capture device of the work site;

wherein the processor is programmed to determine positions of the one or more user specified points in the work site by translating the one or more points indicated on the screen of the display to the image capturing perspective of the image capture device.

12. The apparatus of claim 1, wherein the processor is programmed to cause the information of the translated positions of the one or more user specified points to be graphically displayed on the display by causing a three-dimensional distance map to be displayed on the display, wherein the three-dimensional distance map indicates distances to the one or more user specified points in the target reference frame.

13. The apparatus of claim 12, wherein the processor is programmed to cause graphical representations of the plurality of tools of varying geometries to be displayed on the display so as to indicate a reachability of each of the plurality of tools to the translated positions of the one or more user specified points.

14. The apparatus of claim 1, wherein the processor is programmed to cause the information of the translated positions of the one or more user specified points to be graphically displayed on the display by displaying a distance histogram on the display, wherein the distance histogram includes distances to the one or more user specified points in the target reference frame.

15. The apparatus of claim 14, wherein the processor is programmed to cause graphical representations of extendable lengths of the plurality of tools of varying geometries to be displayed on the histogram for visual comparison to the distances to the one or more user specified points in the target reference frame.

16. A method for providing work site measurements, the method comprising:

determining positions of one or more user specified points in a work site relative to an image capturing perspective of an image capture device;

translating the determined positions of the one or more user specified points so as to be relative to a target reference frame which is noncoincident with the image capturing perspective of the image capture device;

graphically displaying information of the translated positions of the one or more user specified points on a display so as to provide work site measurements relative to the target reference frame; and

displaying information of a plurality of tools of varying geometries along with the information of the translated positions of the one or more user specified points on the display, wherein the work site is accessible by each of the plurality of tools by the tool being inserted through an entry port located at an origin of the target reference frame.

17. The method of claim 16, further comprising:

determining an optimal one of the plurality of tools for use at the work site according to the geometries of the plurality of tools and distances to the translated positions of the one or more user specified points relative to the target reference frame; and

displaying an indication of the optimal one of the plurality of tools on the display.

18. The method of claim 16, wherein the work site comprises one or more objects, and wherein the one or more user specified points are disposed on a surface topology of the one or more objects from the image capturing perspective of the image capture device.

19. An apparatus comprising:

a display;

an image capture device disposed to capture images of a work site; and

a processor programmed to;

determine positions of one or more user specified points in the work site relative to an image capturing perspective of the image capture device;

translate the determined positions of the one or more user specified points so as to be relative to a target reference frame which is noncoincident with the image capturing perspective of the image capture device;

cause information of the translated positions of the one or more user specified points to be graphically displayed on the display so as to provide work site measurements relative to the target reference frame;

determine an optimal one of the plurality of tools for use at the work site according to the geometries of the plurality of tools and distances to the translated positions of the one or more user specified points relative to the target reference frame; and

cause an indication of the optimal one of the plurality of tools to be displayed on the display.

20. The apparatus of claim 19, wherein the optimal one of the plurality of tools is determined by one of:

comparing an unextended length of each of the plurality of tools against a minimum distance of the distances to the translated positions of the one or more user specified points relative to the target reference frame;

comparing an unextended length of each of the plurality of tools against an average distance of the distances to the translated positions of the one or more user specified points relative to the target reference frame; and

comparing an extended length of each of the plurality of tools against a maximum distance of the distances to the translated positions of the one or more user specified points relative to the target reference frame.