Methods and systems for determining and modeling admissible gripper forces for robotic devices

• US 9,393,693 B1
• Filed: 08/25/2014
• Issued: 07/19/2016
• Est. Priority Date: 07/10/2014
• Status: Active Grant

• Current Assignee: Intrinsic Innovation LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Kalakrishnan, Mrinal, Rublee, Ethan
• Primary Examiner(s): Trammell, James
• Assistant Examiner(s): Mott, Adam
• Application Number:
  US14/467,348
• Time in Patent Office: 694 Days
• US Class Current: 1/1
• CPC Class Codes: B25J 9/1602 characterised by the control system, structure, architecture
• CPC Class Codes: B25J 9/1633 compliant, force, torque control, e.g. combined with position control
• CPC Class Codes: B25J 9/1671 characterised by simulation, either to verify existing program or to create and verify new program, CAD/CAM oriented, graphic oriented programming systems
• CPC Class Codes: G05B 2219/39332 Adaptive force control
• CPC Class Codes: G05B 2219/40006 Placing, palletize, un palletize, paper roll placing, box stacking
• CPC Class Codes: G05B 2219/40521 Alternative, allowable path substitution if arm movements not possible
• CPC Class Codes: Y10S 901/02 Arm motion controller

Abstract

Methods and systems for determining and modeling admissible gripper forces are described. An example method may involve receiving data representative of a plurality of trajectories along which a robotic manipulator, while gripping one or more physical objects, previously moved the one or more objects without dropping the one or more objects. The method may also involve determining set of force vectors corresponding to the trajectories. The method may also involve determining a three-dimensional virtual model of the set, where boundaries of the model represent constraints on magnitudes of forces that are applied in a given direction to perform the trajectories. The method may then involve determining one or more subsequent trajectories that correspond to a subsequent set of force vectors that are within the boundaries of the model and along which the robotic manipulator can move a subsequent physical object at particular accelerations without dropping the subsequent object.

Claims

1. A system comprising:

a robotic manipulator;

a gripper coupled to the robotic manipulator;

at least one processor; and

data storage comprising instructions executable by the at least one processor to cause the system to perform operations comprising;

receiving data representative of a plurality of trajectories along which the robotic manipulator, while gripping one or more physical objects using the gripper, previously moved the one or more physical objects without dropping the one or more physical objects,

determining a set of force vectors corresponding to the plurality of trajectories,

determining a virtual model of the set of force vectors, wherein boundaries of the virtual model are representative of constraints on magnitudes of forces that are applied in a given direction to perform the plurality of trajectories, and

determining one or more subsequent trajectories corresponding to a subsequent set of force vectors that are within the boundaries of the virtual model, wherein the determined one or more subsequent trajectories represent trajectories that enable the robotic manipulator to move a subsequent physical object at particular accelerations along the subsequent trajectories without dropping the subsequent physical object.

2. The system of claim 1, wherein the virtual model of the set of force vectors comprises an n-dimensional shape containing the set of force vectors, wherein n is greater than or equal to 2.

3. The system of claim 1, wherein the virtual model of the set of force vectors comprises an n-dimensional convex hull of the set of force vectors, wherein n is greater than or equal to 2.

4. The system of claim 1, wherein receiving data representative of the plurality of trajectories along which the robotic manipulator, while gripping the one or more physical objects using the gripper, previously moved the one or more physical objects without dropping the one or more physical objects comprises:

receiving data representative of a plurality of trajectories along which the robotic manipulator, while gripping the one or more physical objects using the gripper, previously moved the one or more physical objects, and

removing, from the data representative of the plurality of trajectories along which the robotic manipulator, while gripping the one or more physical objects using the gripper, previously moved the one or more physical objects, data representative of a set of trajectories along which the robotic manipulator, while gripping the one or more physical objects using the gripper, previously moved the one or more physical objects and dropped the one or more physical objects.

5. The system of claim 1, the operations further comprising:

receiving additional data, the additional data being representative of a plurality of trajectories along which the robotic manipulator, while gripping the one or more physical objects using the gripper, previously moved the one or more physical objects and dropped the one or more physical objects;

making a comparison between the plurality of trajectories where the robotic manipulator previously dropped the one or more physical objects while moving the one or more physical objects and the plurality of trajectories where the robotic manipulator previously moved the one or more physical objects without dropping the one or more physical objects; and

based on an output of the comparison,

determining a plurality of regions within the virtual model, and

determining respective confidence scores for the plurality of regions, wherein the respective confidence scores are representative of a ratio of a number of trajectories where the robotic manipulator previously dropped the one or more physical objects while moving the one or more physical objects in the plurality of regions to a number of trajectories where the robotic manipulator previously moved the one or more physical objects in the plurality of regions without dropping the one or more physical objects,

wherein determining the one or more subsequent trajectories corresponding to the subsequent set of force vectors that are within the boundaries of the virtual model comprises determining, based on the respective confidence scores, the one or more subsequent trajectories corresponding to the subsequent set of force vectors that are within boundaries of one or more of the determined regions of the virtual model.

6. The system of claim 1, wherein determining the one or more subsequent trajectories comprises:

receiving an instruction for the robotic manipulator to move the subsequent physical object along one or more candidate trajectories corresponding to a candidate set of force vectors, wherein the candidate set of force vectors represent forces to be exerted on the subsequent physical object at a plurality of points in time as the subsequent physical object is moved along the one or more candidate trajectories,

in response to receiving the instruction, making a determination of whether members of the candidate set of force vectors are within the boundaries of the virtual model,

based on the determination, making an adjustment to members of the candidate set of force vectors that are outside of the boundaries of the virtual model,

based on the adjustment, determining an updated candidate set of force vectors, and

determining one or more trajectories that correspond to the updated candidate set of force vectors to be the one or more subsequent trajectories.

7. The system of claim 1, further comprising a force-torque sensor, wherein the set of force vectors are determined by the force-torque sensor.

8. The system of claim 7, wherein determining the set of force vectors and determining the virtual model are based in part on an offset between a location of the force-torque sensor and a location at which the gripper contacts the one or more physical objects.

9. A method comprising:

controlling, by one or more processors, a robotic manipulator to, while gripping one or more physical objects using a gripper, move one or more physical objects along a plurality of trajectories;

receiving, by the one or more processors, data representative of a set of the plurality of trajectories, the set of trajectories including trajectories along which the robotic manipulator, while gripping the one or more physical objects using the gripper, moved the one or more physical objects without dropping the one or more physical objects;

determining, by the one or more processors, a set of force vectors corresponding to the set of trajectories;

determining, by the one or more processors, a virtual model of the set of force vectors, wherein boundaries of the virtual model are representative of constraints on magnitudes of forces that are applied in a given direction to perform the set of trajectories;

determining, by the one or more processors, one or more subsequent trajectories corresponding to a subsequent set of force vectors that are within the boundaries of the virtual model, wherein the determined one or more subsequent trajectories represent trajectories that enable the robotic manipulator to move a subsequent physical object at particular accelerations along the subsequent trajectories without dropping the subsequent physical object; and

controlling, by the one or more processors, the robotic manipulator to move the subsequent physical object along the determined one or more subsequent trajectories.

10. The method of claim 9, wherein the virtual model of the set of force vectors is an isosurface containing the set of force vectors.

11. The method of claim 9, wherein the set of force vectors are determined based on masses of the one or more physical objects and based on respective accelerations corresponding to the set of trajectories.

12. The method of claim 9, wherein receiving data representative of the set of trajectories along which the robotic manipulator, while gripping the one or more physical objects using the gripper, moved the one or more physical objects without dropping the one or more physical objects comprises:

receiving data representative of the plurality of trajectories along which the robotic manipulator, while gripping the one or more physical objects using the gripper, previously moved the one or more physical objects, and

removing, from the data representative of the plurality of trajectories along which the robotic manipulator, while gripping the one or more physical objects using the gripper, previously moved the one or more physical objects, data representative of a second set of the plurality of trajectories along which the robotic manipulator, while gripping the one or more physical objects using the gripper, previously moved the one or more physical objects and dropped the one or more physical objects.

13. The method of claim 9, further comprising:

receiving additional data, the additional data being representative of a second set of the plurality of trajectories along which the robotic manipulator, while gripping the one or more physical objects using the gripper, previously moved the one or more physical objects and dropped the one or more physical objects;

making a comparison between the second set of trajectories where the robotic manipulator previously dropped the one or more physical objects while moving the one or more physical objects and the set of trajectories where the robotic manipulator previously moved the one or more physical objects without dropping the one or more physical objects; and

based on an output of the comparison, determining a plurality of regions within the virtual model,

wherein determining the one or more subsequent trajectories corresponding to the subsequent set of force vectors that are within the boundaries of the virtual model comprises determining the one or more subsequent trajectories corresponding to the subsequent set of force vectors that are within boundaries of one or more of the determined regions of the virtual model.

14. The method of claim 13, further comprising:

based on an output of the comparison, determining respective confidence scores for the plurality of regions, wherein the respective confidence scores are representative of a ratio of a number of trajectories where the robotic manipulator previously dropped the one or more physical objects while moving the one or more physical objects in the plurality of regions to a number of trajectories where the robotic manipulator previously moved the one or more physical objects in the plurality of regions without dropping the one or more physical objects,

wherein determining the one or more subsequent trajectories corresponding to the subsequent set of force vectors that are within boundaries of one or more of the determined regions of the virtual model is based on the respective confidence scores.

15. A non-transitory computer readable medium having stored thereon instructions that, upon execution by a computing device, cause the computing device to perform operations comprising:

controlling a robotic manipulator to, while gripping one or more physical objects using a gripper, move one or more physical objects along a plurality of trajectories;

receiving data representative of a set of the plurality of trajectories, the set of trajectories including trajectories along which the robotic manipulator, while gripping the one or more physical objects using the gripper, moved the one or more physical objects without dropping the one or more physical objects;

determining a set of force vectors corresponding to the set of trajectories;

determining a virtual model of the set of force vectors, wherein boundaries of the virtual model are representative of constraints on magnitudes of forces that are applied in a given direction to perform the set of trajectories;

determining one or more subsequent trajectories corresponding to a subsequent set of force vectors that are within the boundaries of the virtual model, wherein the determined one or more subsequent trajectories represent trajectories that enable the robotic manipulator to move a subsequent physical object at particular accelerations along the subsequent trajectories without dropping the subsequent physical object; and

controlling the robotic manipulator to move the subsequent physical object along the determined one or more subsequent trajectories.

16. The non-transitory computer readable medium of claim 15, wherein determining the one or more subsequent trajectories corresponding to the subsequent set of force vectors that are within the boundaries of the virtual model is performed in response to determining that a location of a center of mass of the subsequent physical object with respect to the gripper substantially matches a location of a center of mass of the one or more physical objects with respect to the gripper.

17. The non-transitory computer readable medium of claim 15, the operations further comprising:

determining a set of torque vectors corresponding to the set of trajectories,

wherein the virtual model includes a virtual model of the set of torque vectors, wherein boundaries of the virtual model are further representative of constraints on magnitudes of torques that are experienced by the one or more physical objects in a given direction to perform the plurality of trajectories.

18. The non-transitory computer readable medium of claim 15, wherein the data further includes respective confidence scores corresponding to at least one force vector of the set of force vectors.

19. The non-transitory computer readable medium of claim 15, wherein the data includes respective accelerations corresponding to the set of trajectories.

20. The non-transitory computer readable medium of claim 19, wherein determining the set of force vectors corresponding to the set of trajectories comprises determining the set of force vectors corresponding to the set of trajectories based at least in part on the respective accelerations corresponding to the set of trajectories.