MACHINE LEARNING A FEATURE DETECTOR USING SYNTHETIC TRAINING DATA

• US 20190325264A1
• Filed: 04/24/2018
• Published: 10/24/2019
• Est. Priority Date: 04/24/2018
• Status: Active Grant

• Current Assignee: HERE Global B.V. (HERE Holding Corporation)
• Original Assignee: HERE Global B.V. (HERE Holding Corporation)
• Inventors: Keserich, Brad, O'Hara, Stephen, Dronen, Nicholas
• Granted Patent: US 10,839,262 B2
• Time in Patent Office: Days
• CPC Class Codes: G01C 21/3602 Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
• CPC Class Codes: G01C 21/3841 Data obtained from two or more sources, e.g. probe vehicles
• CPC Class Codes: G01C 21/3881 Tile-based structures
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06N 3/088 Non-supervised learning, e.g. competitive learning
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 17/20 Finite element generation, e.g. wire-frame surface description, tesselation
• CPC Class Codes: G06V 10/774 Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle

Abstract

Synthetic training information/data of a second probe style is generated based on first probe information/data of a first probe style using a style transfer model. First probe information/data is defined. An instance of first probe information/data comprises labels and first probe style sensor information/data. A style transfer model generates training information/data based on at least a portion of the first probe information/data. An instance of training information/data corresponds to an instance of first probe information/data and comprises second probe style sensor information/data. The first and second probe styles are different. A second probe style model is trained using machine learning and the training information/data. The second probe style model is used to analyze second probe style second probe information/data to extract map information/data from the second probe information/data. Each instance of second probe data is captured by one or more second probe sensors of a second probe apparatus.

Claims

1. A method comprising:

defining, via a network apparatus, first probe data, an instance of first probe data comprising one or more labels and sensor data of a first probe style;

generating, via a style transfer model operating at least in part on the network apparatus, training data based on at least a portion of the first probe data, wherein an instance of training data corresponds to an instance of first probe data and sensor data in a second probe style, the second probe style being different from the first probe style;

training, via the network apparatus, a second probe style model using machine learning and at least a portion of the training data, wherein (a) the second probe style model is used to analyze one or more instances of second probe data of the second probe style to extract map information from the second probe data and (b) each instance of second probe data is captured by one or more second probe sensors of a second probe apparatus.

2. The method of claim 1, wherein the second probe style corresponds to characteristics of the one or more second probe sensors.

3. The method of claim 1, wherein the style transfer model is a generative adversarial network.

4. The method of claim 1, wherein the instance of first probe data is an annotated instance of data of the first probe style and is captured by one or more first probe sensors of a first probe apparatus.

5. The method of claim 1, further comprising generating the instance of first probe data by capturing synthetic sensor data with a virtual sensor within a virtual environment.

6. The method of claim 5, further comprising automatically generating one or more labels for the instance of first probe data based on the virtual environment.

7. The method of claim 1, wherein (a) two or more instances of training data correspond to the same instance of first probe data and (b) the two or more instances of training data are different in at least one feature.

8. The method of claim 1, wherein (a) an instance of training data is an image, (b) the one or more second probe sensors comprise an imaging device configured to capture an image of the second probe style, and (c) the second probe style model is a feature detection model used to generate map information for a digital map.

9. The method of claim 1, further comprising:

receiving, by the network apparatus, at least one instance of second probe data comprising sensor data (a) captured by one or more second probe sensors and (b) of a second probe style;

analyzing the at least one instance of second probe data using the second probe style model to extract map information from the at least one instance of second probe data;

updating, by the network apparatus, a digital map based on the map information; and

providing, by the network apparatus, one or more tiles of the digital map to one or more vehicle apparatuses, wherein at least one of the one or more vehicle apparatuses performs at least one navigation function using the one or more tiles of the digital map.

10. An apparatus comprising at least one processor, a communications interface configured for communicating via at least one network, and at least one memory storing computer program code, the at least one memory and the computer program code configured to, with the processor, cause the apparatus to at least:

define first probe data, an instance of first probe data comprising one or more labels and sensor data of a first probe style;

generate, via a style transfer model operating at least in part on the network apparatus, training data based on at least a portion of the first probe data, wherein an instance of training data corresponds to an instance of first probe data and sensor data in a second probe style, the second probe style being different from the first probe style;

train a second probe style model using machine learning and at least a portion of the training data, wherein (a) the second probe style model is used to analyze one or more instances of second probe data of the second probe style to extract map information from the second probe data and (b) each instance of second probe data is captured by one or more second probe sensors of a second probe apparatus.

11. The apparatus of claim 10, wherein the second probe style corresponds to characteristics of the one or more second probe sensors.

12. The apparatus of claim 10, wherein the style transfer model is a generative adversarial network.

13. The apparatus of claim 10, wherein the instance of first probe data is an annotated instance of data of the first probe style and is captured by one or more first probe sensors of a first probe apparatus.

14. The apparatus of claim 10, wherein the at least one memory and the computer program code are further configured to, with the processor, cause the apparatus to at least generate the instance of first probe data by capturing synthetic sensor data with a virtual sensor within a virtual environment.

15. The apparatus of claim 14, wherein the at least one memory and the computer program code are further configured to, with the processor, cause the apparatus to at least automatically generate one or more labels for the instance of first probe data based on the virtual environment.

16. The apparatus of claim 14, wherein (a) two or more instances of training data correspond to the same instance of first probe data and (b) the two or more instances of training data are different in at least one feature.

17. The apparatus of claim 14, wherein (a) an instance of training data is an image, (b) the one or more second probe sensors comprise an imaging device configured to capture an image of the second probe style, and (c) the second probe style model is a feature detection model used to generate map information for a digital map.

18. The apparatus of claim 14, wherein the at least one memory and the computer program code are further configured to, with the processor, cause the apparatus to at least:

receive at least one instance of second probe data comprising sensor data (a) captured by one or more second probe sensors and (b) of a second probe style;

analyze the at least one instance of second probe data using the second probe style model to extract map information from the at least one instance of second probe data;

update a digital map based on the map information; and

provide one or more tiles of the digital map to one or more vehicle apparatuses, wherein at least one of the one or more vehicle apparatuses performs at least one navigation function using the one or more tiles of the digital map.

19. A computer program product comprising at least one non-transitory computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions comprising executable portions configured, when executed by a processor of a network apparatus, to cause the network apparatus to:

define first probe data, an instance of first probe data comprising one or more labels and sensor data of a first probe style;

generate, via a style transfer model operating at least in part on the network apparatus, training data based on at least a portion of the first probe data, wherein an instance of training data corresponds to an instance of first probe data and sensor data in a second probe style, the second probe style being different from the first probe style;

train a second probe style model using machine learning and at least a portion of the training data, wherein (a) the second probe style model is used to analyze one or more instances of second probe data of the second probe style to extract map information from the second probe data and (b) each instance of second probe data is captured by one or more second probe sensors of a second probe apparatus.

20. The computer program product of claim 19, wherein the second probe style corresponds to characteristics of the one or more second probe sensors.