Lane boundary detection data generation in virtual environment

• US 10,453,256 B2
• Filed: 11/29/2018
• Issued: 10/22/2019
• Est. Priority Date: 10/16/2015
• Status: Active Grant

• Current Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Original Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Inventors: Micks, Ashley Elizabeth, Nallapa, Venkatapathi Raju, Reiff, Brielle, Nariyambut Murali, Vidya, Kadetotad, Sneha
• Primary Examiner(s): Sheng, Xin
• Application Number:
  US16/204,553
• Publication Number:
  US 20190096128A1
• Time in Patent Office: 327 Days
• Field of Search: None
• CPC Class Codes: G06F 18/217 Validation; Performance evaluation; Active pattern learning techniques
• CPC Class Codes: G06F 30/15 Vehicle, aircraft or watercraft design
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06V 10/776 Validation; Performance evaluation
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G09B 9/00 Simulators for teaching or training purposes for the use of weapons F41; computing aspects G06; protocols for games, networked simulations or virtual reality H04L67/131
• CPC Class Codes: G09B 9/042 providing simulation in a real vehicle G09B9/052, G09B9/058 take precedence

Abstract

A method and an apparatus pertaining to generating training data. The method may include executing a simulation process. The simulation process may include traversing one or more virtual sensors over a virtual driving environment defining a plurality of lane markings or virtual objects that are each sensible by the one or more virtual sensors. During the traversing, each of the one or more virtual sensors may be moved with respect to the virtual driving environment as dictated by a vehicle-dynamic model modeling motion of a vehicle driving on a virtual road surface of the virtual driving environment while carrying the one or more virtual sensors. Virtual sensor data characterizing the virtual driving environment may be recorded. The virtual sensor data may correspond to what an actual sensor would produce in a real-world environment that is similar or substantially matching the virtual driving environment.

Claims

1. A method, comprising:

generating, by a processor via a computer simulation, a virtual driving environment;

positioning, by the processor via the computer simulation, one or more virtual sensors within the virtual driving environment, the one or more virtual sensors modeled by the computer simulation and not comprising any physical component;

traversing, by the processor via the computer simulation, the one or more virtual sensors within the virtual driving environment; and

adjusting, by the processor according to a set of bias parameters via the computer simulation, the data to account for a weather condition, a time of a day having different lighting conditions, sensor aging and vehicle aging.

2. The method of claim 1, wherein the virtual driving environment comprises a virtual road surface having one or more driving lanes.

3. The method of claim 2, wherein the virtual road surface further comprises a plurality of lane markings corresponding to the one or more driving lanes, each of the plurality of lane markings sensible by the one or more virtual sensors.

4. The method of claim 2, wherein the virtual driving environment further comprises a plurality of virtual objects distributed therewithin, each of the virtual objects either stationary or mobile relative to the virtual driving environment, and each of the virtual objects sensible by the one or more virtual sensors.

5. The method of claim 4, wherein the data comprises data useful for inferring a location of each of the one or more driving lanes of the virtual road surface as perceived by the one or more virtual sensors sensing one or more of the plurality of virtual objects.

6. The method of claim 5, wherein the data further comprises an annotation characterizing the location of at least one of the one or more driving lanes according to a spatial definition of the at least one of the one or more driving lanes on the virtual road surface.

7. The method of claim 1, wherein the one or more virtual sensors comprise a virtual camera, and wherein the data comprises one or more virtual images of the virtual driving environment as perceived by the virtual camera.

8. The method of claim 1, wherein the one or more virtual sensors comprise a virtual light-detection-and-ranging (LIDAR) device, and wherein the data comprises virtual lane boundaries as perceived by the LIDAR device.

9. The method of claim 1, wherein the positioning comprises setting a spatial relation for each of the one or more virtual sensors with respect to the virtual driving environment according to a vehicle-stationary model modeling, via the computer simulation, a location of the respective virtual sensor on a virtual vehicle carrying the one or more virtual sensors and driving on a virtual road surface of the virtual driving environment.

10. The method of claim 1, wherein the traversing comprises moving each of the one or more virtual sensors with respect to the virtual driving environment according to a vehicle-dynamic model modeling, via the computer simulation, motions of a virtual vehicle carrying the one or more virtual sensors and driving on a virtual road surface of the virtual driving environment.

11. The method of claim 1, wherein the data comprises data characterizing a location of each of one or more driving lanes of a virtual road surface as perceived by the one or more virtual sensors sensing, via the computer simulation, one or more of a plurality of lane markings corresponding to the one or more driving lanes.

12. The method of claim 11, wherein the data further comprises an annotation characterizing the location of at least one of the one or more driving lanes according to a spatial definition of the at least one of the one or more driving lane on the virtual road surface.

13. The method of claim 1, wherein each of the one or more virtual sensors is associated with one of a plurality of sensor types, and wherein each of the plurality of sensor types is modeled by a respective sensor model via the computer simulation.

14. The method of claim 13, wherein the adjusting of the data comprises biasing the respective sensor model according to a set of bias parameters accounting for the weather condition, the time of the day having the different lighting conditions, the sensor aging and the vehicle aging.

15. The method of claim 1, further comprising:

recording, by the processor, data generated by the computer simulation, the data characterizing the virtual driving environment and corresponding to information generated by the one or more virtual sensors sensing the virtual driving environment during the traversing.