Mobile robot that emulates pedestrian walking behavior

• US 10,386,839 B2
• Filed: 05/26/2017
• Issued: 08/20/2019
• Est. Priority Date: 05/26/2016
• Status: Expired due to Fees

• Current Assignee: Boston Incubator Center, LLC, East Japan Railway Company
• Original Assignee: Boston Incubator Center, LLC
• Inventors: Asada, Haruhiko Harry, Weaver, Kota, Lloren-Bonilla, Baldin, Liu, Sheng, Maruyama, Jun
• Primary Examiner(s): Trivedi, Atul
• Application Number:
  US15/607,325
• Publication Number:
  US 20180107217A1
• Time in Patent Office: 816 Days
• Field of Search: 701501
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0223 involving speed control of the vehicle vehicle fittings for automatically controlling, i.e. preventing speed from exceeding an arbitrarily established velocity or maintaining speed at a particular velocity, as selected by the vehicle operator B60K31/00
• CPC Class Codes: G08G 1/005 including pedestrian guidance indicator

Abstract

Methods and systems for navigating a mobile robot through a crowded pedestrian environment based on a trained navigation model are described herein. The trained navigation model receives measurement data identifying nearby pedestrians and the velocity of each nearby pedestrian relative to the mobile robot, and the current position and desired endpoint position of the mobile robot in the crowded pedestrian environment. Based on this information, the trained navigation model generates command signals that cause the mobile robot to adjust its velocity. By repeatedly sampling the velocities of surrounding pedestrians and current location, the navigation model directs the mobile robot toward the endpoint location with a minimum of disruption to the pedestrian traffic flows. The navigation model is trained to emulate desirable pedestrian walking behaviors in a crowded pedestrian environment by tracking the movement of a behavioral trainer through a crowded pedestrian environment.

Claims

1. A system comprising:

a mobile robot including one or more actuators that cause the mobile robot to move at a desired velocity;

one or more sensor systems disposed on the mobile robot that generate a first set of signals indicative of the presence of a plurality of pedestrians nearby the mobile robot and a second set of signals indicative of a location of the mobile robot within a crowded pedestrian environment;

a mobile sensor based crowd analysis module configured to estimate a velocity of each of the plurality of pedestrians relative to the mobile robot at a sample time based on the first set of signals; and

a navigation module configured to estimate a value of the desired velocity at the sample time based on the location of the mobile robot within the crowded pedestrian environment at the sample time, the velocity of each of the plurality of pedestrians relative to the mobile robot at the sample time, and a desired endpoint location of the mobile robot within the crowded pedestrian environment, wherein the determining of the value of the desired velocity is based on a trained navigation model.

2. The system of claim 1, further comprising:

a navigation model training module configured to train the navigation model based on a set of pedestrian walking maneuvers, wherein each pedestrian walking maneuver involves tracking a movement of a behavioral trainer through a crowded pedestrian environment from a particular starting location to a particular endpoint location, wherein a position and velocity of nearby pedestrians relative to the behavioral trainer and a position and velocity of the behavioral trainer are periodically measured during each pedestrian walking maneuver.

3. The system of claim 2, wherein the behavioral trainer is a mobile robot navigated through the crowded pedestrian environment by a human.

4. The system of claim 2, wherein the behavioral trainer is a human who navigates through the crowded pedestrian environment.

5. The system of claim 2, wherein the position and velocity of nearby pedestrians relative to the behavioral trainer and the position and velocity of the behavioral trainer are periodically measured by the one or more sensor systems disposed on the mobile robot.

6. The system of claim 2, further comprising:

one or more sensor systems fixed to one or more locations in the crowded pedestrian environment, wherein the position and velocity of nearby pedestrians relative to the behavioral trainer and the position and velocity of the behavioral trainer are periodically measured by the one or more sensor systems fixed to the one or more locations.

7. The system of claim 6, wherein the set of pedestrian walking maneuvers is selected from a larger set of pedestrian walking maneuvers if a difference in direction between a behavioral trainer associated with a particular walking maneuver and each nearby pedestrian is less than a predetermined threshold value during the particular pedestrian walking maneuver.

8. The system of claim 1, wherein the one or more sensor systems includes any of an image capture system, a range finding sensor system, and a LIDAR sensor system.

9. A system comprising:

a mobile robot including one or more actuators that cause the mobile robot to move at a desired velocity;

one or more sensor systems disposed on the mobile robot that generate a first set of signals indicative of the presence of a plurality of pedestrians nearby the mobile robot and a second set of signals indicative of a location of the mobile robot within a crowded pedestrian environment; and

a non-transitory, computer-readable medium comprising instructions that when executed by a computing system cause the computing system to;

estimate a velocity of each of the plurality of pedestrians relative to the mobile robot at a sample time based on the first set of signals; and

estimate a value of the desired velocity at the sample time based on the location of the mobile robot within the crowded pedestrian environment at the sample time, the velocity of each of the plurality of pedestrians relative to the mobile robot at the sample time, and a desired endpoint location of the mobile robot within the crowded pedestrian environment, wherein the determining of the value of the desired velocity is based on a trained navigation model.

10. The system of claim 9, the non-transitory, computer-readable medium further comprising instructions that when executed by the computing system cause the computing system to:

train the navigation model based on a set of pedestrian walking maneuvers, wherein each pedestrian walking maneuver involves tracking a movement of a behavioral trainer through a crowded pedestrian environment from a particular starting location to a particular endpoint location, wherein a position and velocity of nearby pedestrians relative to the behavioral trainer and a position and velocity of the behavioral trainer are periodically measured during each pedestrian walking maneuver.

11. The system of claim 10, wherein the behavioral trainer is a mobile robot navigated through the crowded pedestrian environment by a human.

12. The system of claim 10, wherein the behavioral trainer is a human who navigates through the crowded pedestrian environment.

13. The system of claim 10, wherein the position and velocity of nearby pedestrians relative to the behavioral trainer and the position and velocity of the behavioral trainer are periodically measured by the one or more sensor systems disposed on the mobile robot.

14. The system of claim 10, further comprising:

one or more sensor systems fixed to one or more locations in the crowded pedestrian environment, wherein the position and velocity of nearby pedestrians relative to the behavioral trainer and the position and velocity of the behavioral trainer are periodically measured by the one or more sensor systems fixed to the one or more locations.

15. The system of claim 14, wherein the set of pedestrian walking maneuvers is selected from a larger set of pedestrian walking maneuvers if a difference in direction between a behavioral trainer associated with a particular walking maneuver and each nearby pedestrian is less than a predetermined threshold value during the particular pedestrian walking maneuver.

16. The system of claim 9, wherein the one or more sensor systems includes any of an image capture system, a range finding sensor system, and a LIDAR sensor system.

17. A method comprising:

generating a first set of signals indicative of a presence of a plurality of pedestrians nearby a mobile robot and a second set of signals indicative of a location of the mobile robot within a crowded pedestrian environment; and

estimating a velocity of each of the plurality of pedestrians relative to the mobile robot at a sample time based on the first set of signals; and

estimating a value of a desired velocity at the sample time based on the location of the mobile robot within the crowded pedestrian environment at the sample time, the velocity of each of the plurality of pedestrians nearby the mobile robot at the sample time, and a desired endpoint location of the mobile robot within the crowded pedestrian environment, wherein the determining of the value of the desired velocity is based on a trained navigation model; and

moving the mobile robot at the value of the desired velocity within the crowded pedestrian environment.

18. The method of claim 17, further comprising:

training the navigation model based on a set of pedestrian walking maneuvers, wherein each pedestrian walking maneuver involves tracking a movement of a behavioral trainer through a crowded pedestrian environment from a particular starting location to a particular endpoint location, wherein a position and velocity of nearby pedestrians relative to the behavioral trainer and a position and velocity of the behavioral trainer are periodically measured during each pedestrian walking maneuver.

19. The method of claim 18, wherein the behavioral trainer is a mobile robot navigated through the crowded pedestrian environment by a human.

20. The method of claim 18, wherein the behavioral trainer is a human who navigates through the crowded pedestrian environment.