Detecting common geographic features in images based on invariant components

• US 9,129,163 B2
• Filed: 06/24/2009
• Issued: 09/08/2015
• Est. Priority Date: 06/24/2009
• Status: Active Grant

• Current Assignee: HERE Global B.V. (HERE Holding Corporation)
• Original Assignee: HERE Global B.V. (HERE Holding Corporation)
• Inventors: Chen, Xin
• Primary Examiner(s): AN, SHAWN S
• Application Number:
  US12/490,575
• Publication Number:
  US 20100328462A1
• Time in Patent Office: 2,267 Days
• Field of Search: 348/148, 348/135, 348142-143, 348136-137, 382/219, 382/113, 382/220, 382/190, 382/209, 382103-104
• US Class Current: 1/1
• CPC Class Codes: G01B 11/022 by means of tv-camera scanning
• CPC Class Codes: G01C 21/30 Map- or contour-matching
• CPC Class Codes: G01C 21/3815 Road data
• CPC Class Codes: G01C 21/3837 Data obtained from a single source
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06V 20/582 of traffic signs
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

Systems, devices, features, and methods for detecting common geographic features in images, such as, for example, to develop a navigation database are disclosed. For example, a method of detecting a common text pattern, such as for a road or path sign, from collected images includes collecting a plurality of images of geographic areas along a road or path. An image of the plurality of images is selected. Components that correspond to an object about the road or path in the selected image are determined. In one embodiment, the determined components are independent or invariant to scale of the object. The determined components are compared to reference components in a data library. If the determined components substantially match the reference components, the object in the selected image is identified to be a common pattern, such as for a standard road or path sign, corresponding to the reference components in the data library.

Claims

1. A method of detecting a common road sign from collected images to develop a master navigation database, the method comprising:

collecting a plurality of images of geographic areas along a road;

selecting an image of the plurality of images;

applying an image area restraint that constrains an allowable area of the selected image for identification of object components to an elevated area of the selected image;

determining components that correspond to an object about the road in the allowable area of the selected image, the determined components being independent to scale of the object;

comparing the determined components to reference components in a data library;

when, based on the comparison, the determined components substantially meet a matching threshold with the reference components, identifying the object in the selected image to be a common text pattern of a common road sign corresponding to the reference components in the data library;

modifying a path segment attribute for the road in the master navigation database to include an association with the common road sign based on the identified common text pattern of the common road sign; and

providing a navigation service from the master navigation database according to the modified attribute of the path segment.

2. The method of claim 1, wherein the common text pattern comprises letters spelling SPEED LIMIT or STOP.

3. The method of claim 1, wherein the determined components are substantially independent to brightness of the object.

4. The method of claim 1, wherein the determined components are substantially independent to rotation of the object.

5. The method of claim 1, further comprising:

identifying a desired common road sign text pattern in a reference image of the plurality of images;

determining the reference components that represent the desired common road sign text pattern in the reference image, the reference components being independent to scale and to rotation of the desired common road sign text pattern; and

storing the reference components in the data library.

6. The method of claim 1, wherein determining the components that correspond to the object about the road comprises using a scale-invariant feature transform.

7. The method of claim 1, wherein determining the components that correspond to the object comprises:

determining components that are independent to scale for the entire selected image;

identifying the determined components that correspond to the object in an area about the road based on clustering; and

filtering out the components not part of the identified cluster.

8. The method of claim 1, wherein the selected image further comprises data indicating a geographic position of acquisition of the image, and the path segment having the modified attribute for the road is associated with the geographic position in the master navigation database.

9. The method of claim 1, wherein the comparing further comprises:

applying a color restraint to the desired road object based on a color restraint associated with the reference invariant features.

10. A method of detecting a standard road object in an image, the method comprising:

capturing a plurality of images along roads, the plurality of images captured via a camera mounted on a vehicle;

applying an image area restraint to the plurality of images that constrains an allowable area of the images for identification of object components to an elevated area of the images;

determining scale-invariant features of each of the plurality of images in the allowable area, the scale-invariant features determined as a function of a scale-invariant feature transform;

comparing the determined scale-invariant features with reference scale-invariant features to determine whether a desired road object is included in at least one of the plurality of images, the reference invariant features corresponding to the desired road object;

identifying at least one of the plurality of images determined to include the desired road object as a function of the comparison between the determined scale-invariant features and the reference scale-invariant features, the desired road object including a standard label pattern;

modifying a path segment attribute or node attribute in a navigation database to include an association with the desired road object including the standard label pattern; and

providing a navigation service from the navigation database according to the modified attribute.

11. The method of claim 10, wherein the desired road object comprises a speed limit sign.

12. The method of claim 10, wherein the desired road object comprises a stop sign.

13. The method of claim 10, wherein the reference scale-invariant features correspond to different points along text of the standard label pattern.

14. The method of claim 10, wherein the determined scale-invariant features comprise amplitude extrema points.

15. The method of claim 10, wherein the determined scale-invariant features comprise gradient vectors.

16. The method of claim 10, further comprising:

developing a navigation database based on identifying the at least one of the plurality of images determined to include the desired road object.

17. The method of claim 10, wherein the standard label pattern includes the common text pattern and the common road sign.

18. A method comprising:

capturing a plurality of images along roads, the plurality of images captured via a camera mounted on a vehicle;

applying an image area restraint to the plurality of images that constrains an allowable area of the images for identification of scale invariant features to an elevated area of the images;

determining scale-invariant features of each of the plurality of images within the allowable area, the scale-invariant features determined as a function of a scale-invariant feature transform;

comparing the determined scale-invariant features with reference scale-invariant features to determine whether a desired road object is included in at least one of the plurality of images, the reference invariant features corresponding to the desired road object;

identifying at least one of the plurality of images determined to include the desired road object as a function of the comparison between the determined scale-invariant features and the reference scale-invariant features, the desired road object including a standard label pattern indicating a speed limit;

assigning the speed limit to a path segment in a navigation database according to the desired road object including the standard label pattern indicating the speed limit; and

providing a navigation service from the navigation database according to the standard label pattern indicating the speed limit.