METHOD AND APPARATUS FOR PREDICTING FEATURE SPACE DECAY USING VARIATIONAL AUTO-ENCODER NETWORKS

• US 20200134354A1
• Filed: 10/30/2018
• Published: 04/30/2020
• Est. Priority Date: 10/30/2018
• Status: Active Grant

• Current Assignee: HERE Global B.V. (Nokia Corporation)
• Original Assignee: HERE Global B.V. (Nokia Corporation)
• Inventors: VISWANATHAN, Anirudh
• Granted Patent: US 10,685,252 B2
• Time in Patent Office: Days
• CPC Class Codes: G01C 21/26 specially adapted for navigation in a road network
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G06F 17/11 for solving equations , e.g. nonlinear equations, general mathematical optimization problems optimization specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

An apparatus, method and computer program product are provided for predicting feature space decay using variational auto-encoder networks. Methods may include: receiving a first image of a road segment including a feature disposed along the road segment; applying a loss function to the feature of the first image; generating a revised image, where the revised image includes a weathered iteration of the feature; generating a predicted image using interpolation between the image and the revised image of a partially weathered iteration of the feature; receiving a user image, where the user image is received from a vehicle traveling along the road segment; correlating a feature in the user image to the partially weathered iteration of the feature in the predicted image; and establishing that the feature in the user image is the feature disposed along the road segment.

Claims

1. An apparatus to facilitate autonomous or semi-autonomous control of a vehicle comprising at least one processor and at least one non-transitory memory including computer program code instructions stored thereon, the computer program code instructions configured to, when executed, cause the apparatus to at least:

receive a first image of a road segment comprising a feature disposed along the road segment;

apply a loss function to the feature of the first image;

generate a revised image, wherein the revised image comprises a weathered iteration of the feature;

generate a predicted image, using interpolation between the image and the revised image, of a partially weathered iteration of the feature;

receive a user image, wherein the user image is received from a vehicle traveling along the road segment;

correlate a feature in the user image to the partially weathered iteration of the feature in the predicted image;

establish that the feature in the user image is the feature disposed along the road segment; and

facilitate autonomous control of the vehicle based, at least in part, on the feature in the user image in response to the feature in the user image being established as the feature disposed along the road segment.

2. The apparatus of claim 1, wherein the loss function comprises a negative log-likelihood function.

3. The apparatus of claim 1, wherein the apparatus is further caused to:

receive training data comprising a plurality of training images of a sample road segment and at least one sample feature thereon, wherein the plurality of training images are temporally spaced apart by at least a predetermined amount of time; and

wherein generating a revised image comprising a weathered iteration of the feature comprises generating a revised image based, at least in part, on information learned from the training data.

4. The apparatus of claim 1, wherein the apparatus is further caused to:

receive a new image of the road segment comprising the feature disposed along the road segment, wherein the new image is received at least a predefined period of time after the first image;

correlating the feature of the new image with the partially weathered iteration of the feature in the predicted image; and

revise the loss function based on a difference between the feature of the new image and the partially weathered iteration of the feature in the predicted image.

5. The apparatus of claim 1, wherein the first feature is a lane line, wherein causing the apparatus to facilitate autonomous control of the vehicle based, at least in part, on the feature in the user image in response to the feature in the user image being established as the feature disposed along the road segment comprises causing the apparatus to provide for autonomous control of the vehicle within a lane of the road segment bordered by the lane line.

6. The apparatus of claim 1, wherein causing the apparatus to receive a first image of a road segment comprising a feature disposed along the road segment comprises causing the apparatus to receive the first image of the road segment, and process the first image using a perception module to identify the feature disposed along the road segment.

7. The apparatus of claim 1, wherein the first image is received from a sensor of an autonomous vehicle traveling along the road segment.

8. A computer program product comprising at least one non-transitory computer-readable storage medium having computer-executable program code instructions stored therein, the computer-executable program code instructions comprising program code instructions to:

receive a first image of a road segment comprising a feature disposed along the road segment;

apply a loss function to the feature of the first image;

generate a revised image, wherein the revised image comprises a weathered iteration of the feature;

generate a predicted image using interpolation between the image and the revised image of a partially weathered iteration of the feature;

receive a user image, wherein the user image is received from a vehicle traveling along the road segment;

correlate a feature in the user image to the partially weathered iteration of the feature in the predicted image;

establish that the feature in the user image is the feature disposed along the road segment; and

facilitate autonomous control of the vehicle based, at least in part, on the feature in the user image in response to the feature in the user image being established as the feature disposed along the road segment.

9. The computer program product of claim 8, wherein the loss function comprises a negative log-likelihood function.

10. The computer program product of claim 8, further comprising program code instructions to:

receive training data comprising a plurality of training images of a sample road segment and at least one sample feature thereon, wherein the plurality of training images are temporally spaced apart by at least a predetermined amount of time; and

wherein generating a revised image comprising a weathered iteration of the feature comprises generating a revised image based, at least in part, on information learned from the training data.

11. The computer program product of claim 8, further comprising program code instructions to:

receive a new image of the road segment comprising the feature disposed along the road segment, wherein the new image is received at least a predefined period of time after the first image;

correlate the feature of the new image with the partially weathered iteration of the feature in the predicted image; and

revise the loss function based on a difference between the feature of the new image and the partially weathered iteration of the feature in the predicted image.

12. The computer program product of claim 8, wherein the first feature is a lane line, wherein the program code instructions to facilitate autonomous control of the vehicle based, at least in part, on the feature in the user image in response to the feature in the user image being established as the feature disposed along the road segment comprises program code instructions to provide for autonomous control of the vehicle within a lane of the road segment bordered by the lane line.

13. The computer program product of claim 8, wherein the program code instructions to receive a first image of a road segment comprising a feature disposed along the road segment comprises program code instructions to receive the first image of the road segment, and process the first image using a perception module to identify the feature disposed along the road segment.

14. The computer program product of claim 8, wherein the first image is received from a sensor of an autonomous vehicle traveling along the road segment.

15. A method comprising:

receiving a first image of a road segment comprising a feature disposed along the road segment;

applying a loss function to the feature of the first image;

generating a revised image, wherein the revised image comprises a weathered iteration of the feature;

generating a predicted image using interpolation between the image and the revised image of a partially weathered iteration of the feature;

receiving a user image, wherein the user image is received from a vehicle traveling along the road segment;

correlating a feature in the user image to the partially weathered iteration of the feature in the predicted image;

establishing that the feature in the user image is the feature disposed along the road segment; and

facilitating autonomous control of the vehicle based, at least in part, on the feature in the user image in response to the feature in the user image being established as the feature disposed along the road segment.

16. The method of claim 15, wherein the loss function comprises a negative log-likelihood function.

17. The method of claim 15, further comprising:

receiving training data comprising a plurality of training images of a sample road segment and at least one sample feature thereon, wherein the plurality of training images are temporally spaced apart by at least a predetermined amount of time; and

wherein generating a revised image comprising a weathered iteration of the feature comprises generating a revised image based, at least in part, on information learned from the training data.

18. The method of claim 15, further comprising:

receiving a new image of the road segment comprising the feature disposed along the road segment, wherein the new image is received at least a predefined period of time after the first image;

correlating the feature of the new image with the partially weathered iteration of the feature in the predicted image; and

revising the loss function based on a difference between the feature of the new image and the partially weathered iteration of the feature in the predicted image.

19. The method of claim 15, wherein the first feature is a lane line, wherein facilitating autonomous control of the vehicle based, at least in part, on the feature in the user image in response to the feature in the user image being established as the feature disposed along the road segment comprises providing for autonomous control of the vehicle within a lane of the road segment bordered by the lane line.

20. The method of claim 15, wherein receiving a first image of a road segment comprising a feature disposed along the road segment comprises receiving the first image of the road segment, and processing the first image using a perception module to identify the feature disposed along the road segment.