Planning for unknown objects by an autonomous vehicle

• US 10,234,864 B2
• Filed: 03/07/2017
• Issued: 03/19/2019
• Est. Priority Date: 03/07/2017
• Status: Active Grant

• Current Assignee: Motional AD, LLC (Aptiv PLC)
• Original Assignee: nuTonomy, Inc. (BorgWarner Incorporated)
• Inventors: Frazzoli, Emilio, Qin, Baoxing
• Primary Examiner(s): Black, Thomas G
• Assistant Examiner(s): Lewandroski, Sara J
• Application Number:
  US15/451,703
• Publication Number:
  US 20180259967A1
• Time in Patent Office: 742 Days
• CPC Class Codes: B60W 10/184 with wheel brakes
• CPC Class Codes: B60W 10/20 including control of steering systems
• CPC Class Codes: B60W 2050/0002 Automatic control, details of type of controller or control system architecture
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2552/00 Input parameters relating to infrastructure
• CPC Class Codes: B60W 2552/10 Number of lanes
• CPC Class Codes: B60W 2552/20 Road profile, i.e. the change in elevation or curvature of a plurality of continuous road segments
• CPC Class Codes: B60W 2552/53 Road markings, e.g. lane marker or crosswalk
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 2554/4023 large-size vehicles, e.g. trucks
• CPC Class Codes: B60W 2554/4026 Cycles
• CPC Class Codes: B60W 2554/4029 Pedestrians
• CPC Class Codes: B60W 2554/406 Traffic density
• CPC Class Codes: B60W 2554/802 Longitudinal distance
• CPC Class Codes: B60W 2555/20 Ambient conditions, e.g. wind or rain
• CPC Class Codes: B60W 2720/24 Direction of travel
• CPC Class Codes: B60W 30/08 Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 30/095 Predicting travel path or likelihood of collision
• CPC Class Codes: B60W 30/0953 : the prediction being responsive to vehicle dynamic parameters
• CPC Class Codes: B60W 30/0956 : the prediction being responsive to traffic or environmental parameters
• CPC Class Codes: B60W 30/143 : Speed control B60W30/16 takes precedence
• CPC Class Codes: B60W 50/0097 : Predicting future conditions
• CPC Class Codes: G08G 1/166 : for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

Among other things, a world model is maintained of an environment of a vehicle. A hypothetical object in the environment that cannot be perceived by sensors of the vehicle is included in the world model.

Claims

1. A computer-implemented method comprising:

maintaining a model of an environment of a vehicle, the environment of the vehicle including a first section and a second section, wherein the first section is capable of being perceived by one or more sensors of the vehicle based on sensor conditions, wherein the second section is not capable of being perceived by the one or more sensors of the vehicle based on sensor conditions and the second section is adjacent to a boundary with the first section;

selectively generating a hypothetical object that is not perceived by the one or more sensors of the vehicle within a particular location of the second section of the environment of the vehicle, wherein selectively generating the hypothetical object comprises;

obtaining historical data on objects previously observed in the environment at the particular location;

selecting a type of the hypothetical object based on the historical data;

selecting an attribute of the hypothetical object probabilistically based on the objects previously observed in the environment at the particular location;

determining, based on the historical data and objects of a same selected type previously observed in the environment at the particular location, that the selected type of the hypothetical object at the particular location behaves with particular attributes; and

in response to determining the selected type of the hypothetical object at the particular location behaves with particular attributes, selecting an additional attribute of the hypothetical object probabilistically based on the objects previously observed in the environment at the particular location; and

updating the model with the generated hypothetical object at the particular location in the environment of the vehicle.

2. The method of claim 1, in which the hypothetical object comprises a moving object.

3. The method of claim 1, in which the hypothetical object comprises an object that uses a path of travel from which the vehicle is excluded.

4. The method of claim 1, in which the hypothetical object comprises at least one of:

a second vehicle, a bicycle, a bus, a train, a pedestrian, and an animal.

5. The method of claim 1, wherein selectively generating the hypothetical object comprises selecting a type of the hypothetical object and an attribute of the hypothetical object probabilistically based on objects previously observed in the environment.

6. The method of claim 5, in which the attribute comprises a size or a speed.

7. The method of claim 1, further comprising including in the model known objects in the environment that are perceived by the one or more sensors of the vehicle or are otherwise known.

8. The method of claim 7, in which the hypothetical object and the known objects maintained by the model are in different parts of the environment.

9. The method of claim 1, in which the vehicle comprises an autonomous vehicle.

10. The method of claim 1, in which the first section and the second section are separated by a boundary.

11. The method of claim 10, comprising detecting of the boundary.

12. The method of claim 11, in which detecting of the boundary comprises using data from one or more sensors to distinguish an observable ground from a foreground that obscures a portion of the observable ground.

13. The method of claim 12, in which the one or more sensors comprise sensors of the vehicle or sensors offboard the vehicle or both.

14. The method of claim 1, further comprising:

querying traffic lane information from a road network database; and

updating, based on the traffic lane information, the generated hypothetical object within the model.

15. The method of claim 1, wherein updating the model with the hypothetical object comprises using stored data to infer a possible location of the hypothetical object.

16. The method of claim 1, wherein updating the model with the hypothetical object comprises determining a location of the vehicle based on a road network database and one or more sensors.

17. The method of claim 1, wherein updating the model with the hypothetical object comprises querying traffic lane information from a database and discretizing a traffic lane into discretized points.

18. The method of claim 1, wherein updating the model with the hypothetical object comprises generating an unknown skeleton of discretized points of a lane that cannot be perceived by the one or more sensors of the vehicle.

19. The method of claim 18, wherein updating the model with the hypothetical object comprises (a) generating a representative shape at a discretized point of the unknown skeleton, and (b) evaluating if the representative shape is completely within the second section of the environment.

20. The method of claim 1, wherein updating the model with the hypothetical object treating a representative shape as the hypothetical object.

21. The method of claim 1 wherein updating the model with the hypothetical object comprises applying temporal filtering to determine a location of the hypothetical object.

22. The method of claim 21, in which applying the temporal filtering comprises smoothing an unknown skeleton by a forward propagated unknown skeleton, wherein the forward propagated unknown skeleton is generated by moving forward an old unknown skeleton along a traffic lane.

23. The method of claim 1, wherein updating the model with the hypothetical object comprises associating one or more attributes with the hypothetical object.

24. The method of claim 23, in which the one or more of the attributes are related to a possible motion state of the hypothetical object.

25. The method of claim 24, in which the motion state comprises a stationary condition.

26. The method of claim 24, in which the motion state comprises a moving condition.

27. The method of claim 24, in which the motion state comprises a speed and a moving direction.

28. The method of claim 27, in which the speed is set to less than or equal to a predetermined maximum value.

29. The method of claim 28, in which the predetermined maximum value comprises a speed limit.

30. The method of claim 28, in which the predetermined maximum value comprises a quantity derived from other objects concurrently or previously observed in the environment.

31. The method of claim 28, in which the predetermined maximum value comprises a quantity derived from at least one of historical data, road configuration, traffic rules, an event, a time, and a weather ondition, hem.

32. The method of claim 1, in which the maintaining of the model comprises accessing a database comprising road network information.

33. The method of claim 1, in which the maintaining of the model comprises using data from the one or more sensors.

34. The method of claim 33, in which the one or more sensors comprise a radar sensor.

35. The method of claim 33, in which the one or more sensors comprise a lidar sensor.

36. The method of claim 33, in which the one or more sensors comprise a camera sensor.

37. The method of claim 36, in which the camera sensor comprises a stereo camera sensor.

38. The method of claim 36, in which the camera sensor comprises a monocular camera sensor.

39. The method of claim 1, comprising updating a trajectory for the vehicle based on the model.

40. The method of claim 39, comprising executing the trajectory for the vehicle.