LANE BOUNDARY DETECTION DATA GENERATION IN VIRTUAL ENVIRONMENT

• US 20170109928A1
• Filed: 10/16/2015
• Published: 04/20/2017
• Est. Priority Date: 10/16/2015
• Status: Active Grant

• Current Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Original Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Inventors: Micks, Ashley Elizabeth, Nallapa, Venkatapathi Raju, Reiff, Brielle, Nariyambut Murali, Vidya, Kadetotad, Sneha
• Granted Patent: US 10,176,634 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/217 Validation; Performance evaluation; Active pattern learning techniques
• CPC Class Codes: G06F 30/15 Vehicle, aircraft or watercraft design
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06V 10/776 Validation; Performance evaluation
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G09B 9/00 Simulators for teaching or training purposes for the use of weapons F41; computing aspects G06; protocols for games, networked simulations or virtual reality H04L67/131
• CPC Class Codes: G09B 9/042 providing simulation in a real vehicle G09B9/052, G09B9/058 take precedence

Abstract

A method and an apparatus pertaining to generating training data. The method may include executing a simulation process. The simulation process may include traversing one or more virtual sensors over a virtual driving environment defining a plurality of lane markings or virtual objects that are each sensible by the one or more virtual sensors. During the traversing, each of the one or more virtual sensors may be moved with respect to the virtual driving environment as dictated by a vehicle-dynamic model modeling motion of a vehicle driving on a virtual road surface of the virtual driving environment while carrying the one or more virtual sensors. Virtual sensor data characterizing the virtual driving environment may be recorded. The virtual sensor data may correspond to what an actual sensor would produce in a real-world environment that is similar or substantially matching the virtual driving environment.

Claims

1. A method, comprising:

generating, by a processor, a virtual driving environment;

positioning, by the processor, one or more virtual sensors within the virtual driving environment;

traversing, by the processor, the one or more virtual sensors within the virtual driving environment; and

recording, by the processor, data characterizing the virtual driving environment, the data corresponding to information generated by the one or more virtual sensors sensing the virtual driving environment during the traversing.

2. The method of claim 1, wherein the virtual driving environment comprises a virtual road surface having one or more driving lanes.

3. The method of claim 2, wherein the virtual road surface further comprises a plurality of lane markings corresponding to the one or more driving lanes, each of the plurality of lane markings sensible by the one or more virtual sensors.

4. The method of claim 1, wherein the one or more virtual sensors comprise a virtual camera, and wherein the data comprises one or more virtual images of the virtual driving environment as perceived by the virtual camera.

5. The method of claim 1, wherein the one or more virtual sensors comprise a virtual light-detection-and-ranging (LIDAR) device, and wherein the data comprises virtual lane boundaries as perceived by the LIDAR device.

6. The method of claim 2, wherein the virtual driving environment further comprises a plurality of virtual objects distributed therewithin, each of the virtual objects either stationary or mobile relative to the virtual driving environment, and each of the virtual objects sensible by the one or more virtual sensors.

7. The method of claim 6, wherein the data comprises data useful for inferring a location of each of the one or more driving lanes of the virtual road surface as perceived by the one or more virtual sensors sensing one or more of the plurality of virtual objects.

8. The method of claim 7, wherein the data further comprises an annotation characterizing the location of at least one of the one or more driving lanes according to a spatial definition of the at least one of the one or more driving lanes on the virtual road surface.

9. The method of claim 1, wherein the positioning comprises setting a spatial relation for each of the one or more virtual sensors with respect to the virtual driving environment according to a vehicle-stationary model modeling a location of the respective virtual sensor on a virtual vehicle carrying the one or more virtual sensors and driving on a virtual road surface of the virtual driving environment.

10. The method of claim 1, wherein the traversing comprises moving each of the one or more virtual sensors with respect to the virtual driving environment according to a vehicle-dynamic model modeling motions of a virtual vehicle carrying the one or more virtual sensors and driving on a virtual road surface of the virtual driving environment.

11. The method of claim 1, wherein the data comprises data characterizing a location of each of one or more driving lanes of a virtual road surface as perceived by the one or more virtual sensors sensing one or more of a plurality of lane markings corresponding to the one or more driving lanes.

12. The method of claim 11, wherein the data further comprises an annotation characterizing the location of at least one of the one or more driving lanes according to a spatial definition of the at least one of the one or more driving lane on the virtual road surface.

13. The method of claim 1, wherein each of the one or more virtual sensors is associated with one of a plurality of sensor types, each of the plurality of sensor types modeled by a respective sensor model, the respective sensor model configured to be biased by a respective set of one or more weather parameters.

14. A sensor data generation system, comprising:

one or more processors; and

memory operably connected to the one or more processors, the memory storing;

a virtual driving environment programmed to include a plurality of virtual objects;

a first software model programmed to model a sensor;

a second software model programmed to model stationary characteristics of a vehicle carrying the sensor;

a third software model programmed to model dynamic characteristics of the vehicle carrying the sensor; and

a simulation module programmed to utilize the virtual driving environment, the first software model, the second software model and the third software model to produce an output modeling an output of the sensor in a real-word scenario in which the sensor is mounted on the vehicle when the vehicle is driven in an actual driving environment similar to or matching the virtual driving environment.

15. The sensor data generation system of claim 14, wherein the simulation module comprises a set of bias parameters programmed to bias the output to account for at least a weather condition, a time of a day, sensor aging and vehicle aging.

16. The sensor data generation system of claim 14, wherein the plurality of virtual objects comprise a plurality of lane markings defining one or more driving lanes.

17. The sensor data generation system of claim 14, wherein the memory further stores data corresponding to the output.

18. The sensor data generation system of claim 17, wherein the simulation module is further programmed to annotate the data with an annotation characterizing the location of one or more driving lanes according to a spatial definition of the one or more driving lanes in the virtual driving environment.

19. The sensor data generation system of claim 18, wherein the sensor comprises a virtual camera, and wherein the data comprise one or more virtual images of the virtual driving environment as perceived by the virtual camera.

20. The sensor data generation system of claim 19, wherein the annotation comprises pixel locations of the one or more driving lanes on the one or more virtual images according to the spatial definition of the one or more driving lanes in the virtual driving environment.