Simulation system and methods for autonomous vehicles

• US 20170132334A1
• Filed: 11/05/2015
• Published: 05/11/2017
• Est. Priority Date: 11/05/2015
• Status: Active Grant

• Current Assignee: Zoox Inc. (Amazon.com, Inc.)
• Original Assignee: Zoox Inc. (Amazon.com, Inc.)
• Inventors: Levinson, Jesse Sol, Rege, Ashutosh Gajanan, Sibley, Gabe Thurston
• Granted Patent: US 10,496,766 B2
• Time in Patent Office: Days
• CPC Class Codes: B60W 2050/0064 using a remote, e.g. cordless, transmitter or receiver unit, e.g. remote keypad or mobile phone
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2554/20 Static objects
• CPC Class Codes: B60W 2554/4026 Cycles
• CPC Class Codes: B60W 2554/4029 Pedestrians
• CPC Class Codes: B60W 50/00 Details of control systems for road vehicle drive control not related to the control of a particular sub-unit , e.g. process diagnostic or vehicle driver interfaces
• CPC Class Codes: B60W 60/0011 involving control alternatives for a single driving scenario, e.g. planning several paths to avoid obstacles
• CPC Class Codes: B60W 60/00253 Taxi operations
• CPC Class Codes: B60W 60/0027 using trajectory prediction for other traffic participants
• CPC Class Codes: G01S 13/865 Combination of radar systems with lidar systems
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 13/87 Combinations of radar systems, e.g. primary radar and secondary radar
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 2013/9316 combined with communication equipment with other vehicles or with base stations
• CPC Class Codes: G01S 2013/9322 using additional data, e.g. driver condition, road state or weather data
• CPC Class Codes: G01S 7/4972 Alignment of sensor
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G06F 30/15 Vehicle, aircraft or watercraft design
• CPC Class Codes: G06F 30/20 Design optimisation, verification or simulation optimisation, verification or simulation of circuit designs G06F30/30

Abstract

Various embodiments relate generally to autonomous vehicles and associated mechanical, electrical and electronic hardware, computer software and systems, and wired and wireless network communications to provide an autonomous vehicle fleet as a service. More specifically, systems, devices, and methods are configured to simulate navigation of autonomous vehicles in various simulated environments. In particular, a method may include receiving data representing characteristics of a dynamic object, calculating a classification of a dynamic object to identify a classified dynamic object, identifying data representing dynamic-related characteristics associated with the classified dynamic object, forming a data model of the classified dynamic object, simulating a predicted range of motion of the classified dynamic object in a simulated environment to form a simulated dynamic object, and simulating a predicted response of a data representation of a simulated autonomous vehicle.

Claims

1. A method, comprising:

identifying first data representing characteristics of one or more dynamic objects in one or more of a simulated environment or a physical environment;

determining a classification of a dynamic object based at least in part on the first data;

identifying second data representing dynamic-related characteristics associated with the dynamic object;

generating a data model of the dynamic object based at least in part on one or more of the first data or the second data;

simulating a predicted range of motion of the dynamic object in a simulated environment; and

simulating a predicted response of one or more functions of a simulated autonomous vehicle based at least in part on the predicted range of motion of the dynamic object.

2. The method of claim 1, wherein simulating the predicted response comprises:

calculating a rate of change of distance between the simulated autonomous vehicle and the predicted range of motion of the dynamic object; and

causing the simulated autonomous vehicle to avoid the dynamic object in the simulated environment based at least in part on the calculated rate of change of distance.

3. The method of claim 1, further comprising validating one or more changes to functionality associated with a physical autonomous vehicle based at least in response on simulating the predicted response of the one or more functions of the simulated autonomous vehicle.

4. The method of claim 1, wherein determining the classification of the dynamic object comprises determining that the dynamic object is one of an animal dynamic object, a pedestrian dynamic object, and a motorized vehicle dynamic object and wherein selecting the predicted range of motion based at least in part on the classification of the dynamic object.

5. The method of claim 1, further comprising:

providing, to a computing device associated with a teleoperator, data associated with simulating the predicted response;

receiving teleoperator data from the computing device associated with the teleoperator; and

controlling an operation of one or more of a physical autonomous vehicle or the simulated autonomous vehicle based at least in part on the teleoperator data.

6. The method of claim 4, further comprising randomizing the predicted range of motion based at least in part on the classification of the dynamic object.

7. The method of claim 1, further comprising:

modeling sensor data to characterize a sensor device to form characterized sensor data, wherein the sensor data is modeled using one or more of data obtained by physical sensors in one or more physical environments or simulated sensor data; and

generating data representing a simulated sensor device based at least in part on the characterized sensor data.

8. The method of claim 7, wherein modeling the sensor data comprises adjusting the data representing the simulated sensor device based at least in part on measurement deviations.

9. The method of claim 7, wherein modeling the sensor data comprises:

modeling Lidar sensor data to characterize a Lidar sensor to form characterized Lidar data;

generating data representing a simulated Lidar device based at least in part on the characterized Lidar data;

modeling data representing Lidar measurement deviations associated with the Lidar device; and

adjusting the data representing the simulated Lidar device based at least in part on the Lidar measurement deviations.

10. The method of claim 1, further comprising:

identifying component characteristics associated with the one or more components of the autonomous vehicle;

generating one or more data models of the one or more components based at least in part on the component characteristics; and

simulating operation of the one or more components based at least in part on the one or more data models to predict behavior of the simulated autonomous vehicle.

11. The method of claim 10, further comprising:

accessing an event data model of that includes data representing event characteristics associated with an event;

simulating the event in the simulated environment based at least in part on the event characteristics; and

simulating another predicted response of the one or more functions of the simulated autonomous vehicle.

12. A system comprising:

one or more computing devices including one or more processors, wherein the one or more computing devices are configured to;

receive first data representing characteristics of dynamic objects;

determine a classification of a dynamic object to identify a classified dynamic object based at least in part on the first data;

identify second data representing dynamic-related characteristics associated with the classified dynamic object;

generate a data model of the classified dynamic object based at least in part on the second data representing the dynamic related characteristics of the classified dynamic object;

simulate a predicted range of motion of the classified dynamic object in a simulated environment; and

simulate a predicted response of a data representation of one or more functions of a simulated autonomous vehicle based at least in part on the predicted range of motion of the classified dynamic object.

13. The system of claim 12, wherein the one or more computing devices are further configured to:

execute simulation instructions to cause the simulated autonomous vehicle to perform a simulated maneuver based at least in part on the predicted range of motion of the simulated dynamic object

generate data associated with the simulated autonomous vehicle performing the simulated maneuver; and

provide the data to one or more of a teleoperator.

14. The system of claim 12, wherein the one or more computing devices are further configured to:

model sensor data to characterize a sensor device to form characterized sensor data; and

generate third data representing a simulated sensor device based at least in part on the characterized sensor data.

15. The system of claim 12, wherein the one or more computing devices are further configured to:

identify component characteristics associated with components of an autonomous vehicle;

generate data models of one or more components based at least in part on the component characteristics; and

simulate operation of the one or more components based at least in part on the data models to predict behavior of the simulated autonomous vehicle.

16. The system of claim 12, further comprising:

receive, via a computing device associated with a teleoperator, data associated with controlling a simulated maneuver;

execute simulation instructions to cause the simulated autonomous vehicle to perform the simulated maneuver;

generate data associated with the simulated autonomous vehicle performing the simulated maneuver; and

analyze the data to determine compliance with one or more policies.

17. The system of claim 12, further comprising providing the predicted response to a computing device associated with a physical autonomous vehicle.

18. A non-transitory computer-readable storage medium having computer-executable instructions stored thereupon which, when executed by a computer, cause the computer to perform actions comprising:

determining a classification of a dynamic object based at least in part on data representing characteristics of one or more dynamic objects in one or more environments;

identifying dynamic-related characteristics associated with the dynamic object;

generating a data model of the dynamic object based at least in part on the dynamic-related characteristics; and

simulating one or more events associated with a simulated autonomous vehicle in a simulated environment based at least in part on a predicted range of motion of the dynamic object.

19. The non-transitory computer-readable storage medium of claim 18, wherein simulating the one or more events, comprises:

calculating a rate of change of distance between the simulated autonomous vehicle and the predicted range of motion of the dynamic object; and

calculate one or more trajectories that causes the simulated autonomous vehicle to avoid the dynamic object in the simulated environment based at least in part on the calculated rate of change of distance.

20. The non-transitory computer-readable storage medium of claim 19, wherein the actions further comprise:

providing the one or more trajectories to a computing device associated with a teleoperator;

receiving a selection of one of the one or more trajectories from the computing device associated with the teleoperator; and

performing one or more of simulating a movement of the simulated autonomous vehicle using the selection of the one or more trajectories or causing a physical autonomous vehicle to utilize the selection of the one or more trajectories.

21. The non-transitory computer-readable storage medium of claim 18, wherein the actions further comprise:

receiving, via a computing device associated with a teleoperator, data associated with controlling a simulated maneuver of the simulated autonomous vehicle; and

executing simulation instructions to cause the simulated autonomous vehicle to perform the simulated maneuver.

22. The non-transitory computer-readable storage medium of claim 21, wherein the actions further comprise:

generating data associated with the simulated autonomous vehicle performing the simulated maneuver; and

analyzing the data to determine compliance with one or more policies.

23. The non-transitory computer-readable storage medium of claim 18, wherein the actions further comprise:

modeling sensor data to characterize a sensor device to form characterized sensor data; and

generating data representing a simulated sensor device based at least in part on the characterized sensor data.