Tooltip stabilization

• US 10,618,165 B1
• Filed: 10/31/2018
• Issued: 04/14/2020
• Est. Priority Date: 01/21/2016
• Status: Active Grant

• Current Assignee: Intrinsic Innovation LLC (Alphabet Inc.)
• Original Assignee: X Development LLC (Alphabet Inc.)
• Inventors: Kolb, Michael, Linnell, Jeff, Jules, Anthony Sean
• Primary Examiner(s): Sample, Jonathan L
• Application Number:
  US16/176,102
• Time in Patent Office: 531 Days
• Field of Search: 700245-264, 3185681, 31856811, 318560, 318567, 901 4, 901 29, 901 42, 901 47
• CPC Class Codes: B25J 13/089 Determining the position of the robot with reference to its environment
• CPC Class Codes: B25J 5/007 mounted on wheels
• CPC Class Codes: B25J 9/0096 co-operating with a working support, e.g. work-table
• CPC Class Codes: B25J 9/162 Mobile manipulator, movable base with manipulator arm mounted on it
• CPC Class Codes: B25J 9/1638 compensation for arm bending/inertia, pay load weight/inertia
• CPC Class Codes: B25J 9/1684 Tracking a line or surface by means of sensors
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G01S 13/86 Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
• CPC Class Codes: G01S 13/862 Combination of radar systems with sonar systems
• CPC Class Codes: G01S 13/865 Combination of radar systems with lidar systems
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 15/931 of land vehicles
• CPC Class Codes: G01S 17/06 Systems determining position data of a target
• CPC Class Codes: G01S 17/08 for measuring distance only indirect measurement G01S17/46; active triangulation systems G01S17/48
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G05B 2219/37423 Distance, gap between tool and surface sensor
• CPC Class Codes: G05B 2219/40298 Manipulator on vehicle, wheels, mobile
• CPC Class Codes: Y10S 901/09 Closed loop, sensor feedback controls arm movement

Abstract

Robotic control systems and methods may include providing an end effector tool of a robotic device configured to perform a task on a work surface within a worksite coordinate frame. Unintended movement over time of the end effector tool with respect to the work surface and with respect to the worksite coordinate frame may be determined based on image data indicative of the work surface, first location data indicative of a first location of the end effector tool with respect to the worksite coordinate frame, and second location data indicative of a second location of the end effector tool with respect to the work surface. One or more control signals for the robotic device may be adjusted in order to counteract the unintended movements of the end effector tool with respect to the work surface and worksite coordinate frame.

Claims

1. A computer-implemented method comprising:

receiving, from a robotic device that includes a sensor and an end effector tool, a location that is estimated for the end effector tool using sensor data that is generated by the sensor included on the robotic device;

receiving, from each of first and second global sensors that are not included on the robotic device, a location that is estimated for the end effector tool using sensor data that is generated by the global sensor;

determining an adjustment to apply to the end effector tool using (i) the location that is estimated for the end effector tool using the sensor data that is generated by the sensor included on the robotic device, (ii) the location that is estimated for the end effector tool using the sensor data that is generated by the first global sensor, and (iii) the location that is estimated for the end effector tool using the sensor data that is generated by the second global sensor; and

providing, to the robotic device, an instruction to apply the adjustment to the end effector tool.

2. The method of claim 1, wherein the sensor included on the robotic device has a higher refresh rate and a lower spatial accuracy than both the first and second global sensors.

3. The method of claim 2, wherein the first global sensor has a higher refresh rate and a lower spatial accuracy than the second global sensor.

4. The method of claim 1, wherein the location that is estimated for the end effector tool using the sensor data that is generated by the sensor included on the robotic device is estimated using dead reckoning or optical flow.

5. The method of claim 1, wherein the locations that are estimated for the end effector tool using sensor data that is generated by first and second global sensors comprise locations of the end effector tool with respect to a worksite coordinate frame.

6. The method of claim 1, wherein the location that is estimated for the end effector tool using the sensor data that is generated by the sensor included on the robotic device comprises a location of the end effector tool with respect to a work surface.

7. The method of claim 1, wherein the first global sensor comprises a camera.

8. The method of claim 7, wherein the second global sensor comprises a laser tracker.

9. The method of claim 1, wherein the robotic device further includes markers for use by the second global sensor in generating the sensor data.

10. A system comprising:

one or more computers; and

a non-transitory computer-readable medium coupled to the one or more computers having instructions stored thereon which, when executed by the one or more computers, cause the one or more computers to perform operations comprising;

receiving, from a robotic device that includes a sensor and an end effector tool, a location that is estimated for the end effector tool using sensor data that is generated by the sensor included on the robotic device;

receiving, from each of first and second global sensors that are not included on the robotic device, a location that is estimated for the end effector tool using sensor data that is generated by the global sensor;

determining an adjustment to apply to the end effector tool using (i) the location that is estimated for the end effector tool using the sensor data that is generated by the sensor included on the robotic device, (ii) the location that is estimated for the end effector tool using the sensor data that is generated by the first global sensor, and (iii) the location that is estimated for the end effector tool using the sensor data that is generated by the second global sensor; and

providing, to the robotic device, an instruction to apply the adjustment to the end effector tool.

11. The system of claim 10, wherein the sensor included on the robotic device has a higher refresh rate and a lower spatial accuracy than both the first and second global sensors.

12. The system of claim 11, wherein the first global sensor has a higher refresh rate and a lower spatial accuracy than the second global sensor.

13. The system of claim 10, wherein the location that is estimated for the end effector tool using the sensor data that is generated by the sensor included on the robotic device is estimated using dead reckoning or optical flow.

14. The system of claim 10, wherein the locations that are estimated for the end effector tool using sensor data that is generated by first and second global sensors comprise locations of the end effector tool with respect to a worksite coordinate frame.

15. The system of claim 10, wherein the location that is estimated for the end effector tool using the sensor data that is generated by the sensor included on the robotic device comprises a location of the end effector tool with respect to a work surface.

16. The system of claim 10, wherein the first global sensor comprises a camera.

17. The system of claim 16, wherein the second global sensor comprises a laser tracker.

18. The system of claim 10, wherein the robotic device further includes markers for use by the second global sensor in generating the sensor data.

19. A non-transitory computer storage medium encoded with a computer program, the computer program comprising instructions that when executed by one or more processors cause the one or more processors to perform operations comprising:

receiving, from a robotic device that includes a sensor and an end effector tool, a location that is estimated for the end effector tool using sensor data that is generated by the sensor included on the robotic device;

receiving, from each of first and second global sensors that are not included on the robotic device, a location that is estimated for the end effector tool using sensor data that is generated by the global sensor;

determining an adjustment to apply to the end effector tool using (i) the location that is estimated for the end effector tool using the sensor data that is generated by the sensor included on the robotic device, (ii) the location that is estimated for the end effector tool using the sensor data that is generated by the first global sensor, and (iii) the location that is estimated for the end effector tool using the sensor data that is generated by the second global sensor; and

providing, to the robotic device, an instruction to apply the adjustment to the end effector tool.

20. The medium of claim 19, wherein:

the sensor included on the robotic device has a higher refresh rate and a lower spatial accuracy than both the first and second global sensors, and

the first global sensor has a higher refresh rate and a lower spatial accuracy than the second global sensor.