Planning for unknown objects by an autonomous vehicle

• US 10,095,234 B2
• Filed: 03/07/2017
• Issued: 10/09/2018
• Est. Priority Date: 03/07/2017
• Status: Active Grant

• Current Assignee: Motional AD, LLC (Aptiv PLC)
• Original Assignee: nuTonomy, Inc. (BorgWarner Incorporated)
• Inventors: Frazzoli, Emilio, Qin, Baoxing
• Primary Examiner(s): Kan, Yuri
• Application Number:
  US15/451,747
• Publication Number:
  US 20180259969A1
• Time in Patent Office: 581 Days
• CPC Class Codes: B60W 10/04 including control of propulsion units
• CPC Class Codes: B60W 10/10 including control of change-speed gearings
• CPC Class Codes: B60W 10/184 with wheel brakes
• CPC Class Codes: B60W 10/20 including control of steering systems
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2552/00 Input parameters relating to infrastructure
• CPC Class Codes: B60W 2552/10 Number of lanes
• CPC Class Codes: B60W 2552/20 Road profile, i.e. the change in elevation or curvature of a plurality of continuous road segments
• CPC Class Codes: B60W 2552/53 Road markings, e.g. lane marker or crosswalk
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 2554/802 Longitudinal distance
• CPC Class Codes: B60W 2555/20 Ambient conditions, e.g. wind or rain
• CPC Class Codes: B60W 2555/60 Traffic rules, e.g. speed limits or right of way
• CPC Class Codes: B60W 2556/10 Historical data
• CPC Class Codes: B60W 40/04 Traffic conditions
• CPC Class Codes: G08G 1/163 involving continuous checking
• CPC Class Codes: G08G 1/165 for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

Among other things, a world model is maintained of an environment of a vehicle. A hypothetical object in the environment that cannot be perceived by sensors of the vehicle is included in the world model.

Claims

1. An apparatus comprising:

an autonomous vehicle comprising;

a) steering, acceleration, deceleration, or gear selection devices or combinations of them configured to effect movement of the autonomous vehicle on a road network; and

b) a computer having a processor to execute a process (i) to generate commands to the steering, acceleration, deceleration, or gear selection devices or combinations of them to move the autonomous vehicle in accordance with driving decisions; and

(ii) to update the commands (a) in response to data representing motion characteristics of a hypothetical vehicle being driven on the road network and (b) based on discretized points of a lane that cannot be perceived by one or more sensors.

2. The apparatus of claim 1, in which data representing motion characteristics of a hypothetical vehicle is probabilistically derived based on vehicles previously observed in an environment of the autonomous vehicle.

3. The apparatus of claim 1, in which regions perceivable and unperceivable by the sensors are separated by a boundary.

4. The apparatus of claim 3, in which the computer detects the boundary.

5. The apparatus of claim 4, in which detecting of the boundary is based on using data from the sensors to distinguish an perceivable ground from a foreground that obscures a portion of the ground.

6. The apparatus of claim 5, in which the sensors comprise sensors onboard the autonomous vehicle.

7. The apparatus of claim 5, in which the sensors comprise sensors offboard the autonomous vehicle.

8. The apparatus of claim 1, in which the computer generates the data based on known objects perceived by the sensors.

9. The apparatus of claim 8, in which generating of the data comprises querying traffic lane information from a road network database.

10. The apparatus of claim 8, in which generating of the data comprises using stored data to infer a possible location of the hypothetical vehicle.

11. The apparatus of claim 8, in which generating of the data comprises determining a location of the autonomous vehicle based on a road network database and the sensors.

12. The apparatus of claim 8, in which generating of the data comprises querying traffic lane information from a database and discretizing the traffic lane into discretized points.

13. The apparatus of claim 8, in which generating of the data comprises (a) generating a representative shape at a discretized point of the lane that cannot be perceived by sensors, and (b) evaluating if the representative shape is completely within an unperceivable region.

14. The apparatus of claim 13, in which generating of the data comprises treating the representative shape as the hypothetical vehicle.

15. The apparatus of claim 8, in which generating of the data comprises applying temporal filtering to determine a location of the hypothetical vehicle.

16. The apparatus of claim 15, in which applying the temporal filtering comprises smoothing an unknown skeleton by a forward propagated unknown skeleton, wherein the forward propagated unknown skeleton is generated by moving forward an old unknown skeleton along a traffic lane.

17. The apparatus of claim 8, in which generating of the data comprises associating one or more attributes with the hypothetical vehicle.

18. The apparatus of claim 17, in which the one or more of the attributes are related to a possible motion state of the hypothetical vehicle.

19. The apparatus of claim 18, in which the possible motion state comprises a stationary condition.

20. The apparatus of claim 1, in which the motion characteristics comprise a hypothetical speed being set to less than or equal to a predetermined maximum value.

21. The apparatus of claim 20, in which the predetermined maximum value comprises a speed limit.

22. The apparatus of claim 20, in which the predetermined maximum value comprises a quantity derived from other objects concurrently or previously perceived in the environment.

23. The apparatus of claim 20, in which the predetermined maximum value comprises a quantity derived from historical data, road configuration, traffic rules, an event, a time, a weather condition, or a combination of two or more of them.

24. The apparatus of claim 1, in which the computer accesses to a database comprising road network information.

25. The apparatus of claim 1, in which the sensors comprises a radar sensor.

26. The apparatus of claim 1, in which the sensors comprises a lidar sensor.

27. The apparatus of claim 1, in which the sensors comprise a camera sensor.

28. The apparatus of claim 27, in which the camera sensor comprises a stereo camera sensor.

29. The apparatus of claim 27, in which the camera sensor comprises a monocular camera sensor.

37. The apparatus of claim 1, in which the computer generates the data based on known objects perceived by the sensors.

38. The apparatus of claim 37, in which generating of the data comprises querying traffic lane information from a road network database.

39. The apparatus of claim 37, in which generating of the data comprises using stored data to infer a possible location of the hypothetical vehicle.

40. The apparatus of claim 37, in which generating of the data comprises determining a location of the autonomous vehicle based on a road network database and the sensors.

41. The apparatus of claim 37, in which generating of the data comprises querying traffic lane information from a database and discretizing the traffic lane into discretized points.

42. The apparatus of claim 37, in which generating of the current data comprises generating an unknown skeleton of discretized points of a lane that cannot be perceived by sensors.

43. The apparatus of claim 37, in which generating of the data comprises (a) generating a representative shape at a discretized point of the lane that cannot be perceived by sensors, and (b) evaluating if the representative shape is completely within an unperceivable region.

44. The apparatus of claim 43, in which generating of the data comprises treating the representative shape as the hypothetical vehicle.

45. The apparatus of claim 37, in which applying the temporal filtering comprises smoothing an unknown skeleton by a forward propagated unknown skeleton, wherein the forward propagated unknown skeleton is generated by moving forward an old unknown skeleton along a traffic lane.

46. The apparatus of claim 37, in which generating of the data comprises associating one or more attributes with the hypothetical vehicle.

47. The apparatus of claim 46, in which the one or more of the attributes are related to a possible motion state of the hypothetical vehicle.

48. The apparatus of claim 47, in which the possible motion state comprises a stationary condition.

49. The apparatus of claim 37, in which the motion characteristics comprise a hypothetical speed is being set to less than or equal to a predetermined maximum value.

50. The apparatus of claim 49, in which the predetermined maximum value comprises a speed limit.

51. The apparatus of claim 50, in which the predetermined maximum value comprises a quantity derived from other objects concurrently or previously perceived in the environment.

52. The apparatus of claim 50, in which the predetermined maximum value comprises a quantity derived from historical data, road configuration, traffic rules, an event, a time, a weather condition, or a combination of two or more of them.

53. The apparatus of claim 37, in which the computer accesses to a database comprising road network information.

54. The apparatus of claim 37, in which the sensors comprises a radar sensor.

55. The apparatus of claim 37, in which the sensors comprises a lidar sensor.

56. The apparatus of claim 37, in which the sensors comprise a camera sensor.

57. The apparatus of claim 27, in which the camera sensor comprises a stereo camera sensor.

58. The apparatus of claim 27, in which the camera sensor comprises a monocular camera sensor.

30. An apparatus comprising:

an autonomous vehicle comprising;

a) steering, acceleration, deceleration, or gear selection devices or combinations of them configured to effect movement of the autonomous vehicle on a road network; and

b) a computer having a processor to execute a process (i) to generate commands to the steering, acceleration, deceleration, or gear selection devices or combinations of them to move the autonomous vehicle in accordance with driving decisions; and

(ii) to update the commands (a) in response to data representing motion characteristics of a hypothetical vehicle being driven on the road network and (b) based on temporal filtering to determine a location of the hypothetical vehicle.

31. The apparatus of claim 30, in which data representing motion characteristics of a hypothetical vehicle is probabilistically derived based on vehicles previously observed in an environment of the autonomous vehicle.

32. The apparatus of claim 30, in which regions perceivable and unperceivable by the sensors are separated by a boundary.

33. The apparatus of claim 32, in which the computer detects the boundary.

34. The apparatus of claim 33, in which detecting of the boundary is based on using data from the sensors to distinguish an perceivable ground from a foreground that obscures a portion of the ground.

35. The apparatus of claim 34, in which the sensors comprise sensors onboard the autonomous vehicle.

36. The apparatus of claim 34, in which the sensors comprise sensors offboard the autonomous vehicle.