Feature-based prediction

• US 10,414,395 B1
• Filed: 04/06/2018
• Issued: 09/17/2019
• Est. Priority Date: 04/06/2018
• Status: Active Grant

• Current Assignee: Zoox Inc. (Amazon.com, Inc.)
• Original Assignee: Zoox Inc. (Amazon.com, Inc.)
• Inventors: Sapp, Benjamin John, Yang, Daylen Guang Yu
• Primary Examiner(s): Camby, Richard M
• Application Number:
  US15/947,486
• Publication Number:
  US 20190308620A1
• Time in Patent Office: 529 Days
• Field of Search: None
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 30/0956 the prediction being responsive to traffic or environmental parameters
• CPC Class Codes: G01S 13/00 Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
• CPC Class Codes: G01S 13/86 Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
• CPC Class Codes: G01S 13/862 Combination of radar systems with sonar systems
• CPC Class Codes: G01S 13/865 Combination of radar systems with lidar systems
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 15/931 of land vehicles
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 2013/9318 Controlling the steering
• CPC Class Codes: G01S 2013/93185 Controlling the brakes
• CPC Class Codes: G01S 2013/9319 Controlling the accelerator
• CPC Class Codes: G01S 7/412 based on a comparison between measured values and known or stored values
• CPC Class Codes: G01S 7/417 involving the use of neural networks
• CPC Class Codes: G01S 7/539 using analysis of echo signal for target characterisation; Target signature; Target cross-section
• CPC Class Codes: G05D 1/0221 involving a learning process
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/20 Ensemble learning
• CPC Class Codes: G06N 5/01 Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: G06V 20/56 : exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/584 : of vehicle lights or traffic lights
• CPC Class Codes: G06V 20/588 : Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G08G 1/00 : Traffic control systems for road vehicles arrangement of road signs or traffic signals E01F9/00 ; automatic vehicle control B62D
• CPC Class Codes: G08G 1/0112 : from the vehicle, e.g. floating car data [FCD]
• CPC Class Codes: G08G 1/0129 : for creating historical data or processing based on historical data
• CPC Class Codes: G08G 1/167 : Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

Feature-based prediction is described. In an example, a vehicle can capture sensor data while traversing an environment and can provide the sensor data to computing system(s). The sensor data can indicate event(s), such as a lane change, associated with agent(s) in the environment. The computing system(s) can determine, based on the sensor data, a time associated with the event and can determine features associated with a period of time relative to the time of the event. In an example, the computing system(s) can aggregate the features with additional features associated with other similar events to generate training data and can train, based at least in part on the training data, a machine learned model for predicting new events. In an example, the machine learned model can be transmitted to vehicle(s), which can be configured to alter drive operation(s) based, at least partly, on output(s) of the machine learned model.

Claims

1. A system comprising:

a processor; and

non-transitory computer-readable media storing instructions executable by the processor, wherein the instructions, when executed by the processor, cause the processor to perform actions comprising;

receiving sensor data from a sensor on an autonomous vehicle;

detecting a vehicle in the sensor data;

associating the vehicle with a lane identification;

determining, based at least in part on a change of the lane identification of the vehicle, the occurrence of a cut-in event, the cut-in event having an associated time;

determining sets of features associated with the cut-in event, the sets of features being associated with a period of time that begins before the event and ends after the event;

aggregating the sets of features with additional sets of features associated with additional cut-in events to generate training data;

training, based at least in part on the training data, a machine learned model for predicting cut-in events; and

sending the machine learned model to the autonomous vehicle for predicting the cut-in events by the autonomous vehicle.

2. The system as claim 1 recites, wherein the sensor comprises a light detection and ranging (LIDAR) sensor, a radio detection and ranging (RADAR) sensor, a sound navigation and ranging (SONAR) sensor, a location sensor, an inertial sensor, a camera, a microphone, or an environment sensor.

3. The system as claim 1 recites, wherein a set of features includes one or more of:

an extent of the vehicle associated with the cut-in event;

a pose of the vehicle associated with the past cut-in event;

a velocity of the vehicle;

an acceleration of the vehicle;

a direction of travel of the vehicle;

a distance between the vehicle and a proximate lane;

a width of a lane within which the vehicle is positioned; and

a semantic feature associated with the vehicle or the environment.

4. The system as claim 1 recites, the actions further comprising:

determining, based at least in part on the sensor data, other sets of features associated with straight driving events; and

adding the other sets of features to the training data.

5. The system as claim 1 recites, wherein determining the sets of features associated with the cut-in event comprises determining individual sets of features associated with individual samples within the period of time.

6. A method comprising:

receiving sensor data from a sensor on a vehicle in an environment, the sensor data indicating a first event associated with an object in the environment;

determining, based at least in part on the sensor data, a time associated with the first event;

determining a feature associated with a period of time relative to the time;

aggregating the feature with an additional feature associated with a second event to generate training data;

training, based at least in part on the training data, a machine learned model for predicting new events; and

transmitting the machine learned model to the vehicle, the vehicle configured to adapt driving operations based at least in part on an output of the machine learned model.

7. The method as claim 6 recites, wherein the feature comprises:

an extent of the object;

a pose of the object;

a velocity of the object;

an acceleration of the object;

a direction of travel of the object;

or

a semantic feature associated with the object or the environment.

8. The method as claim 6 recites, wherein determining the feature comprises:

accessing map data associated with a map of the environment; and

determining, based at least in part on the map data, the feature.

9. The method as claim 6 recites, wherein the period of time begins before the first event and ends after the first event.

10. The method as claim 9 recites, further comprising:

dividing the period of time into a number of samples; and

determining individual sample features for individual samples of the number of samples, the feature corresponding to a sample feature of the individual sample features.

11. The method as claim 6 recites, further comprising:

determining, based at least in part on the sensor data, a negative feature associated with a negative event; and

adding the negative feature to the training data.

12. The method as claim 6 recites, wherein the machine learned model comprises a gradient boosted decision tree.

13. The method as claim 6 recites, wherein the event comprises a cut-in or crossing a crosswalk.

14. The method as claim 6 recites, wherein the object is an additional vehicle, and wherein the event is a cut-in event, the method further comprising:

associating, based at least in part on the map data, a lane identification with the additional vehicle;

monitoring the lane identification for a change; and

determining, based at least in part on the change, the event.

15. A non-transitory computer-readable medium having a set of instructions that, when executed, cause one or more processors to perform operations comprising:

receiving sensor data from a sensor disposed about a vehicle in an environment, the sensor data indicative of an event associated with an object in the environment;

determining, based at least in part on the sensor data, a time associated with the event;

determining a set of features associated with a period of time relative to the time;

aggregating the set of features with an additional set of features associated with an additional event to generate training data;

training, based at least in part on the training data, a machine learned model for predicting new events; and

transmitting the machine learned model to the vehicle, the vehicle configured to alter a drive operation based, at least in part, on an output of the machine learned model.

16. The non-transitory computer-readable medium as claim 15 recites, the operations further comprising:

accessing map data associated with a map of the environment; and

determining at least one feature of the set of features based at least in part on the map data.

17. The non-transitory computer-readable medium as claim 15 recites, wherein the set of features comprises:

an extent of the object;

a pose of the object;

a velocity of the object;

an acceleration of the object;

a direction of travel of the object;

or

a semantic feature associated with the object or the environment.

18. The non-transitory computer-readable medium as claim 15 recites, wherein determining the set of features comprises:

dividing the period of time into a number of samples, each sample comprising a portion of the sensor data associated with the period of time;

determining a sample feature for each sample; and

determining, based at least in part on the sample feature determined for each sample, the set of features.

19. The non-transitory computer-readable medium as claim 15 recites, wherein the object is an additional vehicle, and further wherein the event is a cut-in event, the method further comprising:

associating a lane identification with the additional vehicle;

determining a change of the lane identification; and

determining, based at least in part on the change, the event.

20. The non-transitory computer-readable medium as claim 15 recites, wherein the event comprises a cut-in event or a crosswalk event, and further wherein the machine learned model comprises a gradient boosted decision tree.