Geometrically appropriate tool selection assistance for determined work site dimensions

• US 9,987,751 B2
• Filed: 06/10/2016
• Issued: 06/05/2018
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Original Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Inventors: Itkowitz, Brandon D., DiMaio, Simon P., Gerbi, Craig R., Rogers, Theodore W., Zhao, Wenyi
• Primary Examiner(s): Abdi, Amara
• Application Number:
  US15/178,858
• Publication Number:
  US 20160279801A1
• Time in Patent Office: 725 Days
• CPC Class Codes: B25J 9/1697 Vision controlled systems

Abstract

A robotic system includes a processor that is programmed to determine and cause work site measurements for user specified points in the work site to be graphically displayed in order to provide geometrically appropriate tool selection assistance to the user. The processor is also programmed to determine an optimal one of a plurality of tools of varying geometries for use at the work site and to cause graphical representations of at least the optimal tool to be displayed along with the work site measurements.

Claims

1. An apparatus comprising:

a display;

a slave manipulator adapted to manipulate a tool about a pivot point;

an image capture device disposed to capture images of a work site; and

a processor programmed to;

receive information of a plurality of user specified points in the work site;

determine positions of the plurality of user specified points relative to an image reference frame having an origin collocated with an image capturing end of the image capture device;

translate the determined positions of the plurality of user specified points from the image reference frame to a target reference frame having an origin collocated with the pivot point; and

cause information of the translated positions of the plurality of user specified points to be graphically displayed on the display so as to indicate distances of the plurality of user specified points to the pivot point.

2. The apparatus of claim 1, wherein the processor is programmed to:

cause information of a plurality of tools of varying geometries to be displayed along with the information of the translated positions of the plurality of user specified points on the display, wherein a replacement tool is selectable among the plurality of tools so that the replacement tool may replace the tool which is manipulatable by the slave manipulator about the pivot point.

3. The apparatus of claim 1, wherein the processor is programmed to:

determine an optimal one of a plurality of tools for use at the work site according to the geometries of the plurality of tools and distances to the translated positions of the plurality of user specified points from the pivot point; and

cause an indication of the optimal one of the plurality of tools to be displayed on the display.

4. The apparatus of claim 1, wherein the work site comprises one or more objects, and wherein the plurality of user specified points are disposed on a surface topology of the one or more objects.

5. The apparatus of claim 4, further comprising:

a retractor that is retracting at least a portion of one of the one or more objects, wherein the plurality of user specified points have been individually specified one-at-a-time by a user on a captured image of the work site that is being displayed on the display while the retractor is so retracting the portion of the one of the one or more objects.

6. The method of claim 4, wherein the work site comprises a cavity, wherein the one or more objects are disposed within the cavity, and further comprising:

an insufflator for insufflating the cavity with a gas, wherein the plurality of user specified points have been individually specified one-at-a-time by a user on a captured image of the work site that is being displayed on the display while the insufflator is so inflating the cavity.

7. The apparatus of claim 1, wherein the processor is programmed to determine the positions of the plurality of user specified points in the work site by determining a depth for each of the plurality of user specified points relative to the image reference frame by analyzing information of light reflected from the one or more objects.

8. The apparatus of claim 1, wherein the image capture device captures a pair of stereo images of the work site, and wherein the processor is programmed to determine the positions of plurality of user specified points in the work site relative to the image reference frame by determining a depth for each of the plurality of user specified points relative to the image capturing end of the image capture device by analyzing the stereo images captured by the image capture device.

9. The apparatus of claim 1, further comprising:

a graphical user interface adapted to provide the information of the plurality of user specified points to the processor as the user individually specifies one-at-a-time the plurality of user specified points on a screen of the display while the screen is displaying images captured by the image capture device of the work site;

wherein the processor is programmed to determine positions of the plurality of user specified points in the work site by translating the information of the plurality of user specified points from a display reference frame to the image reference frame.

10. The apparatus of claim 1, further comprising:

a telestrator adapted to provide the information of the plurality of user specified points to the processor as the user individually specifies one-at-a-time the plurality of user specified points on a screen of the display while the screen of the display is displaying images captured by the image capture device of the work site;

wherein the processor is programmed to determine positions of the plurality of user specified points from a display reference frame to the image reference frame.

11. The apparatus of claim 1, further comprising:

a gaze tracker adapted to provide the information of the plurality of user specified points to the processor as the user individually specifies one-at-a-time the plurality of user specified points by gazing on and selecting the plurality of user specified points on a screen of the display while the screen of the display is displaying images captured by the image capture device of the work site;

wherein the processor is programmed to determine positions of the plurality of user specified points from a display reference frame to the image reference frame.

12. The apparatus of claim 1, wherein the processor is programmed to cause the information of the translated positions of the plurality of user specified points to be graphically displayed on the display by causing a three-dimensional distance map to be displayed on the display, wherein the three-dimensional distance map indicates distances to the plurality of user specified points in the target reference frame.

13. The apparatus of claim 12, wherein the processor is programmed to cause graphical representations of a plurality of tools of varying geometries to be displayed on the display so as to indicate a reachability of each of the plurality of tools to the translated positions of the plurality of user specified points.

14. The apparatus of claim 1, wherein the processor is programmed to cause the information of the translated positions of the plurality of user specified points to be graphically displayed on the display by displaying a distance histogram on the display, wherein the distance histogram includes distances to the plurality of user specified points from the pivot point.

15. The apparatus of claim 14, wherein the processor is programmed to cause graphical representations of extendable lengths of a plurality of tools of different geometries to be displayed on the histogram for visual comparison to the distances to the plurality of user specified points from the pivot point.

16. A method for providing work site measurements, the method comprising:

receiving information of a plurality of user specified points in a work site;

determining positions of a plurality of user specified points in the work site relative to an image reference frame having an origin collocated with an image capturing end of an image capture device;

translating the determined positions of the plurality of user specified points from the image reference frame to a target reference frame having an origin collocated with a pivot point about which a tool is manipulatable by a slave manipulator; and

graphically displaying information of the translated positions of the plurality of user specified points on a display so as to indicate distances of the plurality of user specified points to the pivot point.

17. The method of claim 16, further comprising:

displaying information of a plurality of tools of varying geometries along with the information of the translated positions of the plurality of user specified points on the display, wherein a replacement tool is selectable among the plurality of tools so that the replacement tool may replace the tool which is manipulatable by the slave manipulator about the pivot point.

18. The method of claim 16, further comprising:

determining an optimal one of a plurality of tools for use at the work site according to the geometries of the plurality of tools and distances to the translated positions of the plurality of user specified points from the pivot point; and

displaying an indication of the optimal one of the plurality of tools on the display.

19. The method of claim 16, wherein receiving information of a plurality of user specified points in a work site comprises:

receiving indications of the user specified points on one or more objects at the work site as the image capturing end of the image capturing device is being directed towards different locations on the one or more objects.