Shared control of semi-autonomous vehicles including collision avoidance in multi-agent scenarios

• US 9,216,745 B2
• Filed: 08/12/2014
• Issued: 12/22/2015
• Est. Priority Date: 09/16/2013
• Status: Active Grant

• Current Assignee: ETH Zurich, Disney Enterprises Incorporated (The Walt Disney Company)
• Original Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Inventors: Beardsley, Paul A., Watson, Scott Frazier, Alonso-Mora, Javier
• Primary Examiner(s): LANG, MICHAEL DEAN
• Application Number:
  US14/457,291
• Publication Number:
  US 20150284010A1
• Time in Patent Office: 497 Days
• US Class Current: 1/1
• CPC Class Codes: B60W 2720/00 Output or target parameters relating to overall vehicle dynamics
• CPC Class Codes: B60W 30/08 Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences
• CPC Class Codes: B60W 50/10 Interpretation of driver requests or demands
• CPC Class Codes: G05D 1/0011 associated with a remote control arrangement
• CPC Class Codes: G05D 1/0027 involving a plurality of vehicles, e.g. fleet or convoy travelling fleet control of land vehicles from a control room G05D1/0297; traffic control systems for road vehicles G08G1/00; for marine craft G08G3/00; for aircraft G08G5/00
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G05D 1/0289 with means for avoiding collisions between vehicles vehicle fittings for automatically controlling speed including means for detecting potential obstacles B60K31/0008; avoiding obstacles by action on the steering system B62D; radar, sonar, lidar systems designed for anti-collision purposes G01S13/93, G01S15/93, G01S17/93
• CPC Class Codes: G08G 1/096816 where the complete route is transmitted to the vehicle at once
• CPC Class Codes: G08G 1/096838 where the user preferences are taken into account or the user selects one route out of a plurality
• CPC Class Codes: G08G 1/22 Platooning, i.e. convoy of communicating vehicles
• CPC Class Codes: G08G 5/0013 with a ground station
• CPC Class Codes: G08G 5/0026 located on the ground
• CPC Class Codes: G08G 5/0043 Traffic management of multiple aircrafts from the ground G08G5/003 takes precedence; data processing specially designed for resource management, e.g. scheduling or allocating time, human or machine resources G06Q10/06
• CPC Class Codes: G08G 5/0069 specially adapted for an unmanned aircraft
• CPC Class Codes: G08G 5/0082 for monitoring traffic from a ground station radar or analogous systems specially adapted for traffic control G01S13/91
• CPC Class Codes: G08G 5/045 Navigation or guidance aids, e.g. determination of anti-collision manoeuvers radar or analogous systems specially adapted for anti-collision between aircraft G01S13/933

Abstract

A method for providing shared control over movement of a vehicle within a space. The method includes receiving user input related to a velocity and a direction for the vehicle within the space. The method includes processing the user input to selectively adjust the velocity and the direction desired by the user based on a set of predefined constraints to generate a trajectory for the vehicle for an upcoming time period. The method includes operating drive mechanisms in the vehicle based on the trajectory to move the vehicle from a first position to a second position within the space during the upcoming time period. A grid map defining locations of obstacles in the space may be used to define the trajectory to avoid collisions, and a guidance trajectory may be used to further control movement to achieve a desired throughput and control vehicle movement within particular portions of the space.

Claims

1. A method of providing shared control over movement of a vehicle within a space, comprising:

receiving user input related to a velocity for the vehicle within the space;

processing the user input with a local motion planner selectively adjusting the velocity based on a set of predefined constraints and presence of neighboring vehicles in the space to generate a trajectory for the vehicle in the space;

communicating motion control signals including the trajectory to the vehicle; and

operating drive mechanisms in the vehicle based on the motion control signals to move the vehicle from a first position to a second position within the space for an upcoming time period,

wherein the set of predefined constraints comprises a grid map of the space that defines locations of obstacles in the space,

wherein the processing of the user input includes searching the grid map to identify the obstacles and to define a path from the first position to the second position in the space that avoids the obstacles, and

wherein the obstacles comprise virtual objects displayed in the space or on a monitor in the vehicle.

2. The method of claim 1, wherein the trajectory defines a speed for the vehicle and a direction for movement.

3. The method of claim 1, wherein the set of predefined constraints comprises a guidance trajectory defining a passageway through the space that can be traveled by the vehicle without collision with the obstacles and wherein the trajectory is defined to retain the vehicle within the passageway.

4. The method of claim 3, wherein the passageway comprises a set of interconnected waypoints in the space and wherein an active area in which the vehicle may move is associated with each of the waypoints.

5. The method of claim 4, wherein at least some of the active areas differ in size or shape defining differing areas in which the vehicle may move within the space.

6. The method of claim 1, wherein the set of predefined constraints includes kinematic data for the vehicle and wherein the processing of the user input based on presence of neighboring vehicles includes processing data for collision avoidance for other vehicles moving in the space, velocity predictions for ones of the other vehicles neighboring the vehicle in the space, or predictions of movements of the other vehicles during the upcoming time period.

7. A method for controlling vehicle movement through a drive or air space, comprising:

with a controller, receiving user input related to a user-preferred trajectory for moving the vehicle within the space; and

with the controller, processing the user input to selectively adjust the trajectory based on a set of predefined constraints to generate a control trajectory for the vehicle in the space for an upcoming time period,

wherein the set of predefined constraints comprises a grid map of the space that defines locations of obstacles in the space,

wherein the processing of the user input includes searching the grid map to identify the obstacles and to define a path from the first position to the second position in the space that avoids the obstacles,

wherein the set of predefined constraints comprises a guidance trajectory defining a passageway through the space that can be traveled by the vehicle without collision with the obstacles,

wherein the trajectory is defined to retain the vehicle within the passageway,

wherein the passageway comprises a set of interconnected waypoints in the space,

wherein an active area in which the vehicle may move is associated with each of the waypoints, and

wherein at least some of the active areas differ in size or shape defining differing areas in which the vehicle may move within the space.

8. The method of claim 7, wherein the set of predefined constraints includes kinematic data for the vehicle, data for collision avoidance for other vehicles moving in the space, velocity predictions for ones of the other vehicles neighboring the vehicle in the space, or predictions of movements of the other vehicles during the upcoming time period.

9. A method for controlling a vehicle in an installation, comprising:

receiving a user-generated vehicle motion command for the vehicle;

determining whether the user-generated vehicle motion command implicates a constraint;

when the user-generated vehicle motion command implicates the constraint, combining the user-generated vehicle motion command with a system-generated command to produce a modified vehicle motion command to control movement of the vehicle in the installation; and

when the user-generated vehicle motion command does not implicate the constraint, using the user-generated vehicle motion command to control movement of the vehicle in the installation,

wherein the combining of the user-generated vehicle motion command with the system-generated command to produce the modified vehicle motion command comprises performing an optimization adapted to maintain a perception that the user-generated vehicle motion command is directly controlling the vehicle subject to the constraint,

wherein the constraint comprises at least one of avoiding collision with other vehicles moving in the installation, avoiding collision with static obstacles in the installation, and ensuring a defined motion is realizable by the vehicle, and

wherein the optimization is global whereby motion commands are computed concurrently for all vehicles, including the vehicle, in the installation.

10. The method of claim 9, wherein the constraint comprises a schedule such that the vehicle is guaranteed to be in a specified zone in the installation at a system-determined time or time interval.

11. The method of claim 10, wherein the schedule is defined using a guidance trajectory specifying a trajectory, a time associated with points on the trajectory, an active zone associated with each of the points, and a time interval associated with each of the active zones.

12. The method of claim 11, wherein the constraint comprises ensuring a position of the vehicle lies within one of the active zones within the time interval associated with the one of the active zones.

13. The method of claim 12, wherein the schedule is defined by a second guidance trajectory, wherein the vehicle may be controlled in a selective manner using the guidance trajectory or the second guidance trajectory, and wherein switching between the guidance trajectories is performed in response to optimizing a cost function related to a vehicle schedule based on a current state of the vehicle or is performed in response to user input selecting among a two or more of the active states.

14. The method of claim 12, wherein consistency between the vehicle and the guidance trajectory is maintained by adding a motion component that directs the vehicle towards a point within the guidance trajectory for a next time instant.

15. The method of claim 9, wherein the schedule further defines an orientation for the vehicle within the specified zone.

16. The method of claim 9, wherein the constraint comprises a relationship between the vehicle and at least one other vehicle moving through the installation and wherein the user-generated vehicle motion command is modified according to the relationship.

17. The method of claim 9, wherein the constraint comprises a vehicle position relative to static obstacles in the installation and wherein the user-generated vehicle motion command is modified according to a position of the vehicle in the installation relative to the static obstacles.

18. The method of claim 9, further comprising projecting an image of an object onto a surface in the installation and wherein the modified vehicle motion command is generated based on a proximity of the vehicle to the surface onto which the image is projected.

19. The method of claim 18, wherein the modified vehicle motion command is adjusted when the vehicle passes over the surface onto which the image is projected or wherein the image is a fixed or moving projected image generated in response to user input in the vehicle.

20. The method of claim 9, wherein the optimization is local to each vehicle in the installation based on the user-generated vehicle motion command and a predicted motion of neighboring vehicles of the vehicle.

21. The method of claim 9, wherein, when the modified vehicle motion command does not satisfy the constraint, a responsive motion command is generated to stop the vehicle or to continue movement of the vehicle along a prior trajectory.

22. The method of claim 9, wherein surfaces of the installation are characterized by a pixel map comprising a two-dimensional or three-dimensional grid allowing computation of vehicle motions that do not result in collision with the surfaces.

23. The method of claim 9, wherein the constraint is precomputed with respect to a vehicle controller.

24. The method of claim 9, wherein a solution of the optimization is determined by wave expansion within a convex area defined by half-plane constraints determined to avoid collision with other vehicles until a value is determined that satisfies the constraint to avoid collision with fixed surface of the installation and satisfies the constraint to ensure the defined motion is realizable.

25. A method for controlling a vehicle in an installation, comprising:

receiving a user-generated vehicle motion command for the vehicle;

determining whether the user-generated vehicle motion command implicates a constraint;

when the user-generated vehicle motion command implicates the constraint, combining the user-generated vehicle motion command with a system-generated command to produce a modified vehicle motion command to control movement of the vehicle in the installation; and

when the user-generated vehicle motion command does not implicate the constraint, using the user-generated vehicle motion command to control movement of the vehicle in the installation,

wherein the combining of the user-generated vehicle motion command with the system-generated command to produce the modified vehicle motion command comprises performing an optimization adapted to maintain a perception that the user-generated vehicle motion command is directly controlling the vehicle subject to the constraint,

wherein the constraint comprises at least one of avoiding collision with other vehicles moving in the installation, avoiding collision with static obstacles in the installation, and ensuring a defined motion is realizable by the vehicle, and

wherein a solution of the optimization is determined by wave expansion within a convex area defined by half-plane constraints determined to avoid collision with other vehicles until a value is determined that satisfies the constraint to avoid collision with fixed surface of the installation and satisfies the constraint to ensure the defined motion is realizable.

26. The method of claim 25, wherein the constraint comprises a schedule such that the vehicle is guaranteed to be in a specified zone in the installation at a system-determined time or time interval.

27. The method of claim 26, wherein the schedule is defined using a guidance trajectory specifying a trajectory, a time associated with points on the trajectory, an active zone associated with each of the points, and a time interval associated with each of the active zones.

28. The method of claim 27, wherein the constraint comprises ensuring a position of the vehicle lies within one of the active zones within the time interval associated with the one of the active zones.

29. The method of claim 28, wherein the schedule is defined by a second guidance trajectory, wherein the vehicle may be controlled in a selective manner using the guidance trajectory or the second guidance trajectory, and

wherein switching between the guidance trajectories is performed in response to optimizing a cost function related to a vehicle schedule based on a current state of the vehicle or is performed in response to user input selecting among a two or more of the active states.

31. The method of claim 25, wherein the schedule further defines an orientation for the vehicle within the specified zone.

32. The method of claim 25, wherein the constraint comprises a relationship between the vehicle and at least one other vehicle moving through the installation and wherein the user-generated vehicle motion command is modified according to the relationship.

33. The method of claim 25, wherein the constraint comprises a vehicle position relative to static obstacles in the installation and wherein the user-generated vehicle motion command is modified according to a position of the vehicle in the installation relative to the static obstacles.

34. The method of claim 25, further comprising projecting an image of an object onto a surface in the installation and wherein the modified vehicle motion command is generated based on a proximity of the vehicle to the surface onto which the image is projected.

35. The method of claim 34, wherein the modified vehicle motion command is adjusted when the vehicle passes over the surface onto which the image is projected or wherein the image is a fixed or moving projected image generated in response to user input in the vehicle.

36. The method of claim 25, wherein the optimization is local to each vehicle in the installation based on the user-generated vehicle motion command and a predicted motion of neighboring vehicles of the vehicle.

37. The method of claim 25, wherein, when the modified vehicle motion command does not satisfy the constraint, a responsive motion command is generated to stop the vehicle or to continue movement of the vehicle along a prior trajectory.

38. The method of claim 25, wherein surfaces of the installation are characterized by a pixel map comprising a two-dimensional or three-dimensional grid allowing computation of vehicle motions that do not result in collision with the surfaces.

39. The method of claim 25, wherein the constraint is precomputed with respect to a vehicle controller.

30. The method of 28, wherein consistency between the vehicle and the guidance trajectory is maintained by adding a motion component that directs the vehicle towards a point within the guidance trajectory for a next time instant.

40. A method of providing shared control over movement of a vehicle within a space, comprising:

receiving user input related to a velocity for the vehicle within the space;

processing the user input with a local motion planner selectively adjusting the velocity based on a set of predefined constraints and presence of neighboring vehicles in the space to generate a trajectory for the vehicle in the space;

communicating motion control signals including the trajectory to the vehicle; and

operating drive mechanisms in the vehicle based on the motion control signals to move the vehicle from a first position to a second position within the space for an upcoming time period,

wherein the set of predefined constraints comprises a guidance trajectory defining a passageway through the space that can be traveled by the vehicle without collision with the obstacles and wherein the trajectory is defined to retain the vehicle within the passageway,

wherein the passageway comprises a set of interconnected waypoints in the space,

wherein an active area in which the vehicle may move is associated with each of the waypoints, and

wherein at least some of the active areas differ in size or shape defining differing areas in which the vehicle may move within the space.

41. The method of claim 40, wherein the trajectory defines a speed for the vehicle and a direction for movement.

42. The method of claim 40, wherein the set of predefined constraints includes kinematic data for the vehicle.

43. The method of claim 42, wherein the processing of the user input based on presence of neighboring vehicles includes processing data for collision avoidance for other vehicles moving in the space, velocity predictions for ones of the other vehicles neighboring the vehicle in the space, or predictions of movements of the other vehicles during the upcoming time period.