Detecting ground geographic features in images based on invariant components

• US 8,953,838 B2
• Filed: 06/24/2009
• Issued: 02/10/2015
• Est. Priority Date: 06/24/2009
• Status: Active Grant

• Current Assignee: HERE Global B.V. (Nokia Corporation)
• Original Assignee: HERE Global B.V. (Nokia Corporation)
• Inventors: Chen, Xin, Yang, Xiangheng
• Primary Examiner(s): FUJITA, KATRINA R
• Application Number:
  US12/490,538
• Publication Number:
  US 20100329508A1
• Time in Patent Office: 2,057 Days
• Field of Search: 382/103, 707/803, 701/431, 701/450
• US Class Current: 382/103
• CPC Class Codes: G06V 20/38 Outdoor scenes
• CPC Class Codes: G06V 20/39 Urban scenes
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

Systems, devices, features, and methods for detecting geographic features in images, such as, for example, to develop a navigation database are disclosed. For example, a method of detecting a path marking from collected images includes collecting a plurality of images of geographic areas along a path. An image of the plurality of images is selected. Components that represent an object on the path in the selected image are determined. In one embodiment, the determined components are independent or invariant to scale of the object. The determined components are compared to reference components in a data library. If the determined components substantially meet a matching threshold with the reference components, the object in the selected image is identified to be a path marking corresponding to the reference components in the data library.

Claims

1. A method of detecting a geographic feature on a road in an image to develop a navigation database, the method comprising:

capturing a plurality of images along roads, the plurality of images captured via a camera mounted on a vehicle;

determining scale-invariant features of each of the plurality of images, the scale-invariant features determined as a function of a scale-invariant feature transform;

filtering at least one of the scale-invariant features based on distances between the scale-invariant features within each of the plurality of images;

comparing the filtered scale-invariant features with reference scale-invariant features to determine whether a desired geographic object is included in at least one of the plurality of images, the reference scale-invariant features corresponding to scale-invariant features of a first and a second reference image, wherein the first and the second image each contains the desired geographic object, and the second reference image is captured at an out of plane angle to the first reference image;

identifying at least one of the plurality of images determined to include the desired geographic object as a function of the comparison between the filtered scale-invariant features and the reference scale-invariant features, the desired geographic object being a ground geographic feature on a road in the at least one identified image; and

updating a road segment or node of a navigation database to include the ground geographic feature based on identifying the at least one of the plurality of images determined to include the desired geographic object.

2. The method of claim 1, wherein the desired geographic object comprises a cross-walk pattern on the road.

3. The method of claim 2, wherein the cross-walk pattern comprises a zebra pattern.

4. The method of claim 1, wherein the desired geographic object comprises a road marking on the road.

5. The method of claim 4, wherein the road marking comprises a turn restriction marking, an informational marking, or a marking within a lane.

6. The method of claim 1, wherein the filtered scale-invariant features comprise amplitude extrema points.

7. The method of claim 1, wherein the filtered scale-invariant features comprise gradient vectors.

8. The method of claim 1, wherein determining the scale-invariant gradient vectors comprises determining scale-invariant gradient vectors that are substantially invariant to rotation, scale, and illumination.

9. A method of detecting a path marking from collected images, the method comprising:

collecting a plurality of images of geographic areas along a path;

selecting, by a processor, a first and a second image of the plurality of images, the second image being at an out of plane angle with respect to the first image;

identifying, by the processor, gradient vectors in the first and the second image;

filtering, by the processor, gradient vectors based on distances between the gradient vectors;

determining, by the processor, a cluster of gradient vectors in the first image that represent a ground object on the path in the first image and a cluster of gradient vectors in the second image also representing the ground object, the gradient vectors being independent to scale of the ground object;

storing, on a memory, the determined gradient vectors of the first image and the second image as reference vectors associated with the ground object in a data library;

comparing, by the processor, gradient vectors of a third image to the reference gradient vectors associated with the ground object in the data library;

when, based on the comparison, the cluster of gradient vectors substantially meet a matching threshold with the reference gradient vectors, identifying the ground object in the third image; and

providing the third image to a map developer to update a road segment or node of a navigation database corresponding to a location of the third image.

10. The method of claim 9, wherein the path marking comprises a zebra cross-walk pattern.

11. The method of claim 9, wherein the gradient vectors are independent to rotation of the object.

12. The method of claim 9, wherein the gradient vectors are independent to illumination of the object.

13. The method of claim 9, further comprising:

identifying a desired road feature in a reference image of the plurality of images;

determining the reference gradient vectors that correspond to the desired road feature in the reference image, the reference gradient vectors being independent to scale and to rotation of the desired road feature; and

storing the reference gradient vectors in the data library.

14. The method of claim 9, further comprising:

when the object in the selected image is identified to be the path marking on the path, sending a text file to a map developer, the text file identifying the selected image as an image with the path marking on the path.

15. The method of claim 9,

wherein the providing further comprises highlighting areas of the third image found to have matching gradient vectors.

16. A non-transitory computer-readable medium configured to store a computer program that performs a method of identifying a geographic feature in an image corresponding to a road, the method comprising:

identifying a desired ground feature in a first image of a road, the ground feature being on the road;

identifying the desired ground feature in a second image of the road, wherein the second image is at an out of plane angle with respect to the first image of the road;

determining gradient vectors that correspond to the desired ground feature in the first image and the second, the gradient vectors being scale-invariant components;

storing the gradient vectors, derived from the first image and the gradient vectors derived from the second image in association with the desired ground feature in a navigation library of a map database developer;

selecting a third image representing another road;

determining scale-invariant gradient vectors of the third image;

comparing the determined scale-invariant gradient vectors of the third image with the gradient vectors of the first image or the second image stored in the navigation library to determine when the third image includes a feature corresponding to the desired ground feature;

determining a match between the scale-invariant gradient vectors of the third image and the gradient vectors of the first image or the second image; and

updating a road link or node of a navigation database to include the desired ground feature of the third image.

17. The method of claim 16, wherein the desired ground feature comprises a cross-walk pattern in the first image and the second image, the cross-walk pattern being directly on the road.

18. The method of claim 16, further comprising:

clustering at least some of the determined scale-invariant gradient vectors of the third image; and

filtering the determined scale-invariant gradient vectors of the third image that are not part of the clustered scale-invariant gradient vectors.

19. The method of claim 18, wherein the clustered gradient vectors correspond to an area on the other road in the third image.

20. The method of claim 18, wherein comparing comprises comparing the determined scale-invariant gradient vectors remaining after the filtering with the gradient vectors stored in the navigation library to determine when the other image includes a feature corresponding to the desired ground feature.

21. The method of claim 16, wherein determining the scale-invariant gradient vectors of the desired ground feature comprises determining amplitude extrema in regions of a difference image, the difference image comprising an image generated from a difference between the image of the road and a blurred image of the road.

22. The method of claim 16, wherein the gradient vectors being substantially invariant to rotation and to brightness.

23. The method of claim 16, further comprising:

when it is determined that the other image includes the feature corresponding to the desired ground feature on the road, providing identification of the selected other image to the map database developer.