ROBOT ACTION BASED ON HUMAN DEMONSTRATION

• US 20120143374A1
• Filed: 12/05/2011
• Published: 06/07/2012
• Est. Priority Date: 12/03/2010
• Status: Active Grant

• Current Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Original Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Inventors: MISTRY, Michael, MURAI, Akihiko, YAMANE, Katsu, HODGINS, Jessica Kate
• Granted Patent: US 9,162,720 B2
• Time in Patent Office: Days
• US Class Current: 700/259
• CPC Class Codes: B62D 57/032 with alternately or sequentially lifted supporting base and legs; with alternately or sequentially lifted feet or skid B62D57/024 takes precedence

Abstract

Embodiments of the invention provide an approach for reproducing a human action with a robot. The approach includes receiving data representing motions and contact forces of the human as the human performs the action. The approach further includes approximating, based on the motions and contact forces data, the center of mass (CoM) trajectory of the human in performing the action. Finally, the approach includes generating a planned robot action for emulating the designated action by solving an inverse kinematics problem having the approximated human CoM trajectory as a hard constraint and the motion capture data as a soft constraint.

Claims

1. A method for reproducing a designated action of a human with a robot, comprising:

receiving motion capture data and contact force data obtained from a human performing the designated action;

approximating, from the contact force data and the motion capture data, a center of mass (CoM) trajectory of the human in performing the designated action; and

generating, via one or more processors, a planned robot action for emulating the designated action, wherein generating the planned robot action comprises solving an inverse kinematics problem having the approximated human CoM trajectory and the motion capture data as constraints to determine a plurality of joint angle trajectories for a corresponding plurality of joints on the robot.

2. The method of claim 1, wherein the designated action is a sit-to-stand action.

3. The method of claim 1, wherein the inverse kinematics problem has the approximated human CoM trajectory as a hard constraint and the motion capture data as a soft constraint.

4. The method of claim 1, wherein the inverse kinematics problem further has constraints relative to at least one of matching torques, smoothness, energy efficiency, or collision avoidance in performing the designated action.

5. The method of claim 3, wherein the motion capture data provides trajectories of a plurality of markers placed on the human, and wherein the contact force data represents forces exerted by the human on one or more force sensor plates in performing the designated action.

6. The method of claim 5, wherein approximating the CoM trajectory of the human action comprises:

determining, based on the recorded contact forces, a center of pressure (CoP) trajectory of the human in performing the designated action;

mapping the markers to joint angle data of a human skeletal model;

scaling one or more of a length and mass of a link of the human skeletal model so that the CoP trajectory of the human skeletal model in performing a simulation of the designated action matches, up to a predefined error tolerance, the CoP trajectory of the human in performing the designated action; and

determining a CoM trajectory for the scaled skeletal model in performing the simulation of the designated action.

7. The method of claim 1, wherein generating the planned robot action further comprises estimating a set of inertial parameters of the robot, wherein estimating the set of inertial parameters comprises:

placing the robot in a plurality of static positions;

determining contact forces and joint torques of the robot in each static position; and

estimating the set of inertial parameters based on the determined contact forces and joint torques

8. The method of claim 7, wherein the plurality of static positions comprise one or more of a standing position and a squatting position.

9. The method of claim 7, wherein estimating a set of inertial parameters comprises solving a weighted least squares problem.

10. The method of claim 7, wherein the set of inertial parameters is a reduced set of inertial parameters that is relevant to the action.

11. The method of claim 10, wherein the reduced set of inertial parameters comprises inertial parameters for a reduced model of the robot with motion restricted to a sagittal plane of the robot.

12. The method of claim 11, wherein the reduced model of the robot is a model comprising the following degrees of freedom:

ankle, knee, hip, and torso.

13. A non-transitory computer readable storage medium comprising a program product which, when executed by a processor is configured to perform an operation for reproducing an action of a human with a robot, the operation comprising:

receiving motion capture data and contact force data obtained from a human performing the designated action;

approximating, from the contact force data and the motion capture data, a center of mass (CoM) trajectory of the human in performing the designated action; and

generating, via one or more processors, a planned robot action for emulating the designated action, wherein generating the planned robot action comprises solving an inverse kinematics problem having the approximated human CoM trajectory and the motion capture data as constraints to determine a plurality of joint angle trajectories for a corresponding plurality of joints on the robot.

14. The computer readable storage medium of claim 13, wherein the action is a sit-to-stand action.

15. The computer readable storage medium of claim 13, wherein the inverse kinematics problem has the approximated human CoM trajectory as a hard constraint and the motion capture data as a soft constraint.

16. The computer readable storage medium of claim 13, wherein the inverse kinematics problem further has constraints relative to at least one of matching torques, smoothness, energy efficiency, or collision avoidance in performing the designated action.

17. The computer readable storage medium of claim 15, wherein the motion capture data provides trajectories of a plurality of markers placed on the human, and wherein the contact force data represents forces exerted by the human on one or more force sensor plates in performing the designated action.

18. The computer readable storage medium of claim 17, wherein approximating the CoM trajectory of the human in performing the action comprises:

determining, based on the recorded contact forces, a center of pressure (CoP) trajectory of the human in performing the designated action;

mapping the markers to joint angle data of a human skeletal model;

scaling one or more of a length and mass of a link of the human skeletal model so that the CoP trajectory of the human skeletal model in performing a simulation of the designated action matches, up to a predefined error tolerance, the CoP trajectory of the human in performing the designated action; and

determining a CoM trajectory for the scaled skeletal model in performing the simulation of the designated action.

19. The computer-readable storage medium of claim 13, wherein generating the planned robot action further comprises estimating a set of inertial parameters of the robot, wherein estimating the set of inertial parameters comprises:

placing the robot in a plurality of static positions;

determining contact forces and joint torques of the robot in each static position; and

estimating the set of inertial parameters based on the determined contact forces and joint torques.

20. The computer readable storage medium of claim 19, wherein the set of inertial parameters is a reduced set of inertial parameters that is relevant to the action.

21. The computer readable storage medium of claim 17, wherein the reduced set of inertial parameters comprises inertial parameters for a reduced model of the robot with motion restricted to a sagittal plane of the robot.

22. A system, comprising:

a memory comprising a program; and

a processor which, when executing the program, is configured to;

receive motion capture data and contact force data obtained from a human performing the designated action;

approximate, from the contact force data and the motion capture data, a center of mass (CoM) trajectory of the human in performing the designated action; and

generate, via one or more processors, a planned robot action for emulating the designated action, wherein generating the planned robot action comprises solving an inverse kinematics problem having the approximated human CoM trajectory and the motion capture data as constraints to determine a plurality of joint angle trajectories for a corresponding plurality of joints on the robot.

23. The system of claim 22, wherein the inverse kinematics problem has the approximated human CoM trajectory as a hard constraint and the motion capture data as a soft constraint.

24. The system of claim 23, wherein approximating the CoM trajectory of the human in performing the action comprises:

determining, based on the recorded contact forces, a center of pressure (CoP) trajectory of the human in performing the designated action;

mapping the markers to joint angle data of a human skeletal model;

scaling one or more of a length and mass of a link of the human skeletal model so that the CoP trajectory of the human skeletal model in performing a simulation of the designated action matches, up to a predefined error tolerance, the CoP trajectory of the human in performing the designated action; and

determining a CoM trajectory for the scaled skeletal model in performing the simulation of the designated action.

25. The system of claim 22, wherein generating the planned robot action further comprises estimating a set of inertial parameters of the robot, wherein estimating the set of inertial parameters comprises:

placing the robot in a plurality of static positions;

determining contact forces and joint torques of the robot in each static position; and

estimating the set of inertial parameters based on the determined contact forces and joint torques.

26. The system of claim 25, wherein the set of inertial parameters is a reduced set of inertial parameters that is relevant to the action.

27. The system of claim 25, wherein the reduced set of inertial parameters comprises inertial parameters for a reduced model of the robot with motion restricted to a sagittal plane of the robot.

28. A method for reproducing a designated action of a human with a robot, comprising:

receiving motion capture data and contact force data obtained from a human performing the designated action, wherein the motion capture data comprises trajectories of a set of markers placed on the human;

approximating, from the contact force data and the motion capture data, a center of mass (CoM) trajectory of the human in performing the designated action; and

reproducing the designated action by;

generating, via one or more processors, a planned robot action for emulating the designated action, wherein generating the planned robot action comprises solving an inverse kinematics problem having the approximated human CoM trajectory and the motion capture data as constraints to determine a plurality of joint angle trajectories for a corresponding plurality of joints on the robot; and

sending the joint angle trajectories to a set of joint controllers of the robot to cause the robot to perform the designated action.