METHOD, APPARATUS, AND SYSTEM FOR GENERATING FEATURE CORRESPONDENCE BETWEEN IMAGE VIEWS

• US 20200134844A1
• Filed: 10/26/2018
• Published: 04/30/2020
• Est. Priority Date: 10/26/2018
• Status: Active Grant

• Current Assignee: HERE Global B.V. (Nokia Corporation)
• Original Assignee: HERE Global B.V. (Nokia Corporation)
• Inventors: MITTAL, Anish, CHEN, Zhanwei, KURIAN, Joseph
• Granted Patent: US 11,055,862 B2
• Time in Patent Office: Days
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

An approach is provided for determining a feature correspondence between image views. The approach, for example, involves retrieving a top down image for an area of interest and determining a ground level camera pose path for the area of interest. The approach also involves selecting a portion of the top down image that corresponds to a geographic area within a distance threshold from the ground level camera pose path, and then processing the portion of the top down image to identify a semantic feature. The approach further involves determining a subset of camera poses of the ground level pose path that is within a sphere of visibility of the semantic feature, and retrieving one or more ground level images captured with the subset of camera poses. The approach further involves determining the feature correspondence of the semantic feature between the top down image and the one or more ground level images.

Claims

1. A computer-implemented method for determining a feature correspondence between image views comprising:

retrieving a top down image for an area of interest;

determining a ground level camera pose path for the area of interest;

selecting a portion of the top down image that corresponds to a geographic area within a distance threshold from the ground level camera pose path;

processing the portion of the top down image to identify a semantic feature;

determining a subset of camera poses of the ground level pose path that is within a sphere of visibility of the semantic feature;

retrieving one or more ground level images captured with the subset of camera poses; and

determining the feature correspondence of the semantic feature between the top down image and the one or more ground level images.

2. The method of claim 1, wherein the one or more ground level images are determined based on determining that semantic feature is identified in the one or more ground level image.

3. The method of claim 1, wherein the ground level camera pose path is based on sensor data indicating a trajectory of a ground level camera traveling through the area of interest and a view direction of the ground level camera.

4. The method of claim 3, wherein the trajectory is based on a drive path of a vehicle on which the ground level camera is mounted.

5. The method of claim 1, wherein the sphere of visibility represents an estimated physical volume around the semantic feature in which the semantic feature is photo-identifiable in an image captured by a ground level camera traveling along the ground level pose path.

6. The method of claim 5, further comprising:

computing the estimated physical volume based on a map data of the area of interest.

7. The method of claim 1, wherein the feature correspondence relates a first image location at which the semantic feature is identified in the top down image to a second image location at which the semantic feature is identified in the one or more ground level images.

8. An apparatus for determining a feature correspondence between image views comprising:

at least one processor; and

at least one memory including computer program code for one or more programs, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following,

retrieve a top down image for an area of interest;

determine a ground level camera pose path for the area of interest;

select a portion of the top down image that corresponds to a geographic area within a distance threshold from the ground level camera pose path;

process the portion of the top down image to identify a semantic feature;

determine a subset of camera poses of the ground level pose path that is within a sphere of visibility of the semantic feature;

retrieve one or more ground level images captured with the subset of camera poses; and

determine the feature correspondence of the semantic feature between the top down image and the one or more ground level images.

9. The apparatus of claim 8, wherein the one or more ground level images are determined based on determining that semantic feature is identified in the one or more ground level image.

10. The apparatus of claim 8, wherein the ground level camera pose path is based on sensor data indicating a trajectory of a ground level camera traveling through the area of interest and a view direction of the ground level camera.

11. The apparatus of claim 10, wherein the trajectory is based on a drive path of a vehicle on which the ground level camera is mounted.

12. The apparatus of claim 8, wherein the sphere of visibility represents an estimated physical volume around the semantic feature in which the semantic feature is photo-identifiable in an image captured by a ground level camera traveling along the ground level pose path.

13. The apparatus of claim 12, wherein the apparatus is further caused to:

compute the estimated physical volume based on a map data of the area of interest.

14. The apparatus of claim 8, wherein the feature correspondence relates a first image location at which the semantic feature is identified in the top down image to a second image location at which the semantic feature is identified in the one or more ground level images.

15. A non-transitory computer-readable storage medium for determining a feature correspondence between image views, carrying one or more sequences of one or more instructions which, when executed by one or more processors, cause an apparatus to perform:

retrieving a top down image for an area of interest;

determining a ground level camera pose path for the area of interest;

selecting a portion of the top down image that corresponds to a geographic area within a distance threshold from the ground level camera pose path;

processing the portion of the top down image to identify a semantic feature;

determining a subset of camera poses of the ground level pose path that is within a sphere of visibility of the semantic feature;

retrieving one or more ground level images captured with the subset of camera poses; and

determining the feature correspondence of the semantic feature between the top down image and the one or more ground level images.

16. The non-transitory computer-readable storage medium of claim 15, wherein the one or more ground level images are determined based on determining that semantic feature is identified in the one or more ground level image.

17. The non-transitory computer-readable storage medium of claim 15, wherein the ground level camera pose path is based on sensor data indicating a trajectory of a ground level camera traveling through the area of interest and a view direction of the ground level camera.

18. The non-transitory computer-readable storage medium of claim 17, wherein the trajectory is based on a drive path of a vehicle on which the ground level camera is mounted.

19. The non-transitory computer-readable storage medium of claim 15, wherein the sphere of visibility represents an estimated physical volume around the semantic feature in which the semantic feature is photo-identifiable in an image captured by a ground level camera traveling along the ground level pose path.

20. The non-transitory computer-readable storage medium of claim 19, wherein the apparatus is caused to further perform:

computing the estimated physical volume based on a map data of the area of interest.