Robot motion replanning based on user motion

• US 9,452,530 B2
• Filed: 09/12/2014
• Issued: 09/27/2016
• Est. Priority Date: 09/12/2014
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Inventors: Kim, Young-Ho, Sisbot, Emrah Akin
• Primary Examiner(s): Edwards, Jerrah
• Assistant Examiner(s): Bendidi, Rachid
• Application Number:
  US14/485,491
• Publication Number:
  US 20160075023A1
• Time in Patent Office: 746 Days
• US Class Current: 1/1
• CPC Class Codes: B25J 9/1666 Avoiding collision or forbidden zones
• CPC Class Codes: G01S 19/14 specially adapted for specific applications
• CPC Class Codes: G01S 19/393 Trajectory determination or predictive tracking, e.g. Kalman filtering
• CPC Class Codes: G01S 5/0072 Transmission between mobile stations, e.g. anti-collision systems
• CPC Class Codes: G01S 5/0294 Trajectory determination or predictive filtering, e.g. target tracking or Kalman filtering
• CPC Class Codes: G05B 2219/40476 Collision, planning for collision free path
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 40/25 Recognition of walking or running movements, e.g. gait recognition

Abstract

The disclosure includes a system and method for determining a robot path based on user motion by determining a current position of a robot with a processor-based computing device programmed to perform the determining, receiving sensor readings on positions, directions, and velocities of a visually-impaired user and other users, generating a model of the motions of the visually-impaired user and the other users, the model including a user path for the visually-impaired user and a robot path for the robot, generating a collision prediction map to predict collisions between at least one of the robot, the visually-impaired user, and the other users, determining whether there is a risk of collision for either the visually-impaired user or the robot, and responsive to the risk of collision, updating at least one of the user path and the robot path.

Claims

1. A method comprising:

determining a current position of a robot with a processor of the robot;

receiving sensor readings on positions, directions, and velocities of a visually-impaired user and other users who are not visually-impaired in an environment where the visually-impaired user is known from the other users;

generating a model of motions of the visually-impaired user and the other users, the model including a user path for the visually-impaired user and a robot path for the robot;

generating a collision prediction map to predict collisions between at least one of the robot, the visually-impaired user, and the other users;

determining whether there is a risk of collision for either the visually-impaired user or the robot; and

responsive to the risk of collision, updating at least one of the user path and the robot path based on the risk of collision;

wherein generating the model of the motions further comprises;

receiving torso-directed movement data for the visually-impaired user;

determining a face direction for the visually-impaired user;

determining a walking direction for the visually-impaired user;

determining whether the visually-impaired user exhibits low-consistency movement or high-consistency movement based on the torso direction and the walking direction; and

determining human motion uncertainty based on the consistency of movement.

2. The method of claim 1, further comprising responsive to an absence of the risk of collision, providing directions to the visually-impaired user based on the user path.

3. The method of claim 1, further comprising:

instructing the robot to follow the robot path with the processor; and

providing directions of the visually-impaired user based on the user path.

4. The method of claim 1, wherein generating the model of the motions further comprises:

receiving torso-directed movement data for the other users who are not visually-impaired;

determining the face direction for the other users;

determining the walking direction for the other users;

determining whether the other users exhibit low-consistency movement or high-consistency movement based on the torso direction and the walking direction of the other users; and

determining human motion uncertainty for the other users based on the consistency of movement of the other users.

5. The method of claim 1, further comprising using a probabilistic method to estimate a degree of human reaction by updating each variant scale parameter between a torso direction and current movement.

6. The method of claim 1, wherein generating the collision prediction map further comprises:

generating pairs between the visually-impaired user and each of the other users;

determining a difference of arrival time between members of each pair; and

determining whether the difference of arrival time is less than a boundary parameter.

7. The method of claim 6, wherein responsive to the difference of arrival time being less than the boundary parameter, further comprising:

increasing a collision probability; and

generating the collision map based on the collision probability for each pair.

8. The method of claim 6, wherein responsive to the difference of arrival time being less than the boundary parameter, further comprising:

setting a collision probability to zero; and

generating the collision map based on the collision probability for each pair.

9. A computer program product comprising a non-transitory computer-usable medium including a computer-readable program, wherein the computer-readable program when executed on a computer causes the computer to:

determine a current position of a robot;

receive sensor readings on positions, directions, and velocities of a visually-impaired user and other users in an environment where the visually-impaired user is known from the other users, wherein the other users are not visually-impaired;

generate a model of motions of the visually-impaired user and the other users, the model including a user path for the visually-impaired user and a robot path for the robot;

generate a collision prediction map to predict collisions between at least one of the robot, the visually-impaired user, and the other users;

determine whether there is a risk of collision for either the visually-impaired user or the robot; and

responsive to the risk of collision, update at least one of the user path and the robot path based on the risk of collision;

wherein generating the model of the motions further comprises;

receiving torso-directed movement data for the visually-impaired user;

determining a face direction for the visually-impaired user;

determining a walking direction for the visually-impaired user;

determining whether the visually-impaired user exhibits low-consistency movement or high-consistency movement based on the torso direction and the walking direction; and

determining human motion uncertainty based on the consistency of movement.

10. The computer program product of claim 9, wherein the computer-readable program is further configured to:

generate velocity constraints for the robot to follow a new path; and

instruct the robot to move to a goal position based on the new path and the velocity constraints.

11. The computer program product of claim 9, wherein the computer-readable program is further configured to:

instruct, by the computer, the robot to follow the robot path; and

provide directions of the visually-impaired user based on the user path.

12. The computer program product of claim 9, wherein generating the collision prediction map further comprises:

generating pairs between the visually-impaired user and each of the other users;

determining a difference of arrival time between members of each pair; and

determining whether the difference of arrival time is less than a boundary parameter.

13. The computer program product of claim 12, wherein the computer-readable program is further configured to use a probabilistic method to estimate a degree of human reaction by updating each variant scale parameter between a torso direction and current movement.

14. The computer program product of claim 9, wherein generating the model of motions further comprises:

receiving torso-directed movement data for the other users who are not visually-impaired;

determining the face direction for the other users;

determining the walking direction for the other users;

determining whether the other users exhibit low-consistency movement or high-consistency movement based on the torso direction and the walking direction of the other users; and

determining human motion uncertainty for the other users based on the consistency of movement of the other users.

15. A system comprising:

a processor; and

a non-transitory memory storing instructions that, when executed, cause the system to;

determine a current position of a robot;

receive sensor readings on positions, directions, and velocities of a visually-impaired user and other users in an environment where the visually-impaired user is known from the other users, wherein the other users are not visually-impaired;

generate a model of motions of the visually-impaired user and the other users, the model including a user path for the visually-impaired user and a robot path for the robot;

generate a collision prediction map to predict collisions between at least one of the robot, the visually-impaired user, and the other users;

determine whether there is a risk of collision for either the visually-impaired user or the robot; and

responsive to the risk of collision, update at least one of the user path and the robot path based on the risk of collision;

wherein generating the model of the motions further comprises;

receiving torso-directed movement data for the visually-impaired user;

determining a face direction for the visually-impaired user;

determining a walking direction for the visually-impaired user;

determining whether the visually-impaired user exhibits low-consistency movement or high-consistency movement based on the torso direction and the walking direction; and

determining human motion uncertainty based on the consistency of movement.

16. The system of claim 15, wherein the memory stores instructions that, when executed, cause the system to providing directions to the visually-impaired user based on the user path responsive to an absence of the risk of collision.

17. The system of claim 15, wherein the memory stores instructions that, when executed, cause the system to:

instruct, by the processor, the robot to follow the robot path; and

provide directions of the visually-impaired user based on the user path.

18. The system of claim 15, wherein generating the model of the motions further comprises:

receiving torso-directed movement data for the other users who are not visually-impaired;

determining the face direction for the other users;

determining the walking direction for the other users;

determining whether the other users exhibit low-consistency movement or high-consistency movement based on the torso direction and the walking direction; and

determining human motion uncertainty based on the consistency of movement.

19. The system of claim 18, wherein the memory is further configured to use a probabilistic method to estimate a degree of human reaction by updating each variant scale parameter between a torso direction and current movement.

20. The system of claim 15, wherein generating the collision prediction map further comprises:

generating pairs between the visually-impaired user and each of the other users;

determining a difference of arrival time between members of each pair; and

determining whether the difference of arrival time is less than a boundary parameter.