Tooltip stabilization

• US 9,757,859 B1
• Filed: 01/21/2016
• Issued: 09/12/2017
• Est. Priority Date: 01/21/2016
• Status: Active Grant

• Current Assignee: Intrinsic Innovation LLC (Alphabet Inc.)
• Original Assignee: X Development LLC (Alphabet Inc.)
• Inventors: Kolb, Michael, Linnell, Jeff, Jules, Anthony Sean
• Primary Examiner(s): Sample, Jonathan L
• Application Number:
  US15/003,526
• Time in Patent Office: 600 Days
• CPC Class Codes: B25J 13/089 Determining the position of the robot with reference to its environment
• CPC Class Codes: B25J 5/007 mounted on wheels
• CPC Class Codes: B25J 9/0096 co-operating with a working support, e.g. work-table
• CPC Class Codes: B25J 9/162 Mobile manipulator, movable base with manipulator arm mounted on it
• CPC Class Codes: B25J 9/1638 compensation for arm bending/inertia, pay load weight/inertia
• CPC Class Codes: B25J 9/1684 Tracking a line or surface by means of sensors
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G01S 13/86 Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
• CPC Class Codes: G01S 13/862 Combination of radar systems with sonar systems
• CPC Class Codes: G01S 13/865 Combination of radar systems with lidar systems
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 15/931 of land vehicles
• CPC Class Codes: G01S 17/06 Systems determining position data of a target
• CPC Class Codes: G01S 17/08 for measuring distance only indirect measurement G01S17/46; active triangulation systems G01S17/48
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G05B 2219/37423 Distance, gap between tool and surface sensor
• CPC Class Codes: G05B 2219/40298 Manipulator on vehicle, wheels, mobile
• CPC Class Codes: Y10S 901/09 Closed loop, sensor feedback controls arm movement

Abstract

Robotic control systems and methods may include providing an end effector tool of a robotic device configured to perform a task on a work surface within a worksite coordinate frame. Unintended movement over time of the end effector tool with respect to the work surface and with respect to the worksite coordinate frame may be determined based on image data indicative of the work surface, first location data indicative of a first location of the end effector tool with respect to the worksite coordinate frame, and second location data indicative of a second location of the end effector tool with respect to the work surface. One or more control signals for the robotic device may be adjusted in order to counteract the unintended movements of the end effector tool with respect to the work surface and worksite coordinate frame.

Claims

1. A method comprising:

providing an end effector tool mounted to a moveable component of a robotic device, wherein the end effector tool is configured to perform a task on a work surface in a worksite coordinate frame;

receiving, from a first sensor, image data indicative of the work surface;

determining, based on the image data, first movement information indicating unintended movement over time of the end effector tool with respect to the work surface;

receiving, from a second sensor, location data indicating a location of the end effector tool with respect to the worksite coordinate frame;

determining, based on the location data, second movement information indicating unintended movement over time of the end effector tool with respect to the worksite coordinate frame;

determining, based on the first and second movement information, one or more adjustments to one or more control signals for the robotic device in order to counteract the unintended movements of the end effector tool with respect to the work surface and worksite coordinate frame; and

communicating instructions to the robotic device to operate in accordance with the one or more adjustments.

2. The method of claim 1, further comprising:

receiving, from a third sensor, second location data indicating a second location of the end effector tool with respect to the work surface;

determining, based on the second location data, third movement information indicating unintended movement over time of the end effector tool with respect to the work surface;

determining, based on the third movement information, one or more adjustments to one or more control signals for the robotic device in order to counteract the unintended movement over time of the end effector tool with respect to the work surface; and

communicating instructions to the robotic device to operate in accordance with the one or more adjustments.

3. The method of claim 1, wherein the first sensor comprises an optical flow sensor.

4. The method of claim 3, further comprising receiving, from a laser sensor, distance data indicative of a distance of the work surface from the laser sensor.

5. The method of claim 4, further comprising adjusting the focal length of the optical flow sensor based on the distance data received from the laser sensor.

6. The method of claim 4, further comprising:

determining, based on the distance data received from the laser sensor, fourth movement information indicating unintended movement over time of the end effector tool with respect to the work surface;

determining, based on the fourth movement information, one or more adjustments to one or more control signals for the robotic device in order to counteract the unintended movement of the end effector tool; and

communicating instructions to the robotic device to operate in accordance with the one or more adjustments.

7. The method of claim 1, wherein the second sensor comprises a motion capture sensor, wherein a plurality of motion capture markers are positioned in the work environment, and wherein at least one of the plurality of motion capture markers is attached to the end effector tool.

8. The method of claim 1, wherein the second sensor comprises a laser tracker configured to measure a location of one or more markers attached to the robotic device, wherein the location of the end effector tool is determined with respect to the location of the one or more markers attached to the robotic device.

9. The method of claim 8, wherein at least one of the one or more markers is attached to the end effector tool, wherein the end effector tool performing a task on the work surface interrupts a line of sight between the laser tracker and the at least one of the one or more markers attached to the end effector tool, wherein the robotic device is configured to periodically manipulate the end effector tool to restore the line of sight.

10. The method of claim 2, wherein the third sensor comprises an inertial measurement unit.

11. A system comprising:

an end effector tool mounted to a moveable component of a robotic device, wherein the end effector tool is configured to perform a task on a work surface in a worksite coordinate frame; and

a control system configured to;

receive, from a first sensor, image data indicative of the work surface;

determine, based on the image data, first movement information indicating unintended movement over time of the end effector tool with respect to the work surface;

receive, from a second sensor, location data indicating a location of the end effector tool with respect to the worksite coordinate frame;

determine, based on the location data, second movement information indicating unintended movement over time of the end effector tool with respect to the worksite coordinate frame;

determine, based on the first and second movement information, one or more adjustments to one or more control signals for the robotic device in order to counteract the unintended movements of the end effector tool with respect to the work surface and worksite coordinate frame; and

communicate instructions to the robotic device to operate in accordance with the one or more adjustments.

12. The system of claim 11, wherein the control system is further configured to:

receive, from a third sensor, second location data indicating a second location of the end effector tool with respect to the work surface;

determine, based on the second location data, third movement information indicating unintended movement over time of the end effector tool with respect to the work surface;

determine, based on the third movement information, one or more adjustments to one or more control signals for the robotic device in order to counteract the unintended movement over time of the end effector tool with respect to the work surface; and

communicate instructions to the robotic device to operate in accordance with the one or more adjustments.

13. The system of claim 11, wherein the first sensor comprises an optical flow sensor.

14. The system of claim 13, wherein the control system is further configured to receive, from a laser sensor, distance data indicative of a distance of the work surface from the laser sensor.

15. The system of claim 14, wherein the control system is further configured to adjust the focal length of the optical flow sensor based on the distance data received from the laser sensor.

16. The system of claim 14, wherein the control system is further configured to:

determine, based on the distance data received from the laser sensor, fourth movement information indicating unintended movement over time of the end effector tool with respect to the work surface;

determine, based on the fourth movement information, one or more adjustments to one or more control signals for the robotic device in order to counteract the unintended movement of the end effector tool; and

communicate instructions to the robotic device to operate in accordance with the one or more adjustments.

17. The system of claim 11, wherein the second sensor comprises a motion capture sensor, wherein a plurality of motion capture markers are positioned in the work environment, and wherein at least one of the plurality of motion capture markers is attached to the end effector tool.

18. The system of claim 11, wherein the second sensor comprises a laser tracker configured to measure a location of one or more markers attached to the robotic device, wherein the location of the end effector tool is determined with respect to the location of the one or more markers attached to the robotic device.

19. The system of claim 18, wherein at least one of the one or more markers is attached to the end effector tool, wherein the end effector tool performing a task on the work surface interrupts a line of sight between the laser tracker and the at least one of the one or more markers attached to the end effector tool, wherein the robotic device is configured to periodically manipulate the end effector tool to restore the line of sight.

20. The system of claim 12, wherein the third sensor comprises an inertial measurement unit.