VISION SYSTEM AND METHOD OF ANALYZING AN IMAGE

• US 20120114178A1
• Filed: 11/09/2011
• Published: 05/10/2012
• Est. Priority Date: 11/09/2010
• Status: Active Grant

• Current Assignee: Harman Becker Automotive Systems GmbH (Samsung Electronics Co. Ltd.)
• Original Assignee: Harman Becker Automotive Systems GmbH (Samsung Electronics Co. Ltd.)
• Inventors: Pryakhin, Alexey, Kunath, Peter, Platonov, Juri
• Granted Patent: US 8,929,604 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G01C 21/3602 Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata

Abstract

A vision system comprises a camera that captures an image and a processor coupled to process the received image to determine at least one feature descriptor for the image. The processor includes an interface to access annotated map data that includes geo-referenced feature descriptors. The processor is configured to perform a matching procedure between the at least one feature descriptor determined for the at least one image and the retrieved geo-referenced feature descriptors.

Claims

1. A vehicle vision system, comprising:

a camera configured to capture at least one image; and

a processor coupled to the camera and configured to determine at least one feature descriptor, and in particular a plurality of feature descriptors, for the at least one image; and

the processor having an interface to access annotated map data including geo-referenced feature descriptors to retrieve a plurality of geo-referenced feature descriptors and position information respectively associated with the geo-referenced feature descriptors,

where the processor is configured to perform a matching procedure between the at least one feature descriptor determined for the at least one image and the retrieved geo-referenced feature descriptors.

2. The vehicle vision system of claim 1, where the processor is configured to identify a set of geo-referenced feature descriptors matching a plurality of feature descriptors determined for the at least one image, and to determine three coordinates of a vehicle orientation based on the position information associated with the set of geo-referenced feature descriptors.

3. The vehicle vision system of claim 2, where the three determined coordinates of the vehicle orientation define three angles.

4. The vehicle vision system of claim 3, where for at least some of the geo-referenced feature descriptors, the associated position information includes a height coordinate.

5. The vehicle vision system of claim 2, where the processor is configured to control, in an augmented reality mode, an optical output device based on the three determined coordinates of the vehicle orientation.

6. The vehicle vision system of claim 1, further comprising:

a positioning device configured to determine a vehicle position,

where the processor is coupled to the positioning device and is configured to retrieve the plurality of geo-referenced feature descriptors based on the vehicle position.

7. The vehicle vision system of claim 6, where the processor is configured to estimate a location of an object in the at least one image based on a position information associated with at least one of the retrieved geo-referenced feature descriptors and based on the vehicle position.

8. The vehicle vision system of claim 7, the positioning device being further configured to determine a vehicle orientation, the processor being configured to estimate the location of the object further based on the vehicle orientation.

9. The vehicle vision system of claim 6, where the processor is configured to identify a portion of the at least one image based on the vehicle position and based on position information associated with at least one of the plurality of retrieved geo-referenced feature descriptors,

the processor being configured to selectively determine the at least one feature descriptor for the identified portion of the image.

10. The vehicle vision system of claim 1, where the geo-referenced feature descriptors included in the annotated map data are respectively associated with at least one of a plurality of object categories, and processor is configured to selectively perform the matching between the at least one feature descriptor determined for the at least one image and the retrieved geo-referenced feature descriptors which are associated with a selected object category.

11. The vehicle vision system of claim 1, further comprising an optical output device, the processor being configured to control the optical output device to output information indicating an object in the at least one image which has a feature descriptor matching one of the retrieved geo-referenced feature descriptors.

12. The vehicle vision system of claim 1, where at least one of the geo-referenced feature descriptors comprises an associated position information including a height information, the processor being configured to perform the matching procedure based on the height information.

13. The vehicle vision system of claim 1, further comprising a database that stores the annotated map data, the annotated map data including a plurality of geo-referenced feature vectors respectively formed from a plurality of feature descriptors, each of the geo-referenced feature vectors having an associated position information stored in the database, the processor being configured to determine a feature vector for the at least one image, the determined feature vector including a plurality of feature descriptors, and to match the feature vector determined for the at least one image to a feature vector stored in the database.

14. The vehicle vision system of claim 1, further comprising a wireless communication device coupled to the processor, the processor being configured to transmit a request message via the wireless communication device to request transmission of the plurality of geo-referenced feature descriptors and/or to transmit the at least one feature descriptor determined for the at least one image via the wireless communication device.

15. The vehicle vision system of claim 1, where the at least one feature descriptor determined for the at least one image being selected from a group comprising a Scale-Invariant Feature Transform (SIFT) descriptor, a Speeded Up Robust Features (SURF) descriptor, a Rotation Invariant Feature Transform (RIFT) descriptor, a Generalized Robust Invariant Feature (G-RIF) descriptor, a Principal Components Analysis SIFT (PCA-SIFT) descriptor or a Gradient Location and Orientation Histogram (GLOH) descriptor.

16. A method for analyzing at least one image captured onboard a vehicle, the method comprising:

determining least one feature descriptor, and in particular a plurality of feature descriptors, for the at least one image;

retrieving a plurality of geo-referenced feature descriptors and position information respectively associated with the geo-referenced feature descriptors; and

performing a matching procedure between the at least one feature descriptor determined for the at least one image and the retrieved geo-referenced feature descriptors.

17. The method of claim 16, further comprising:

identifying in the matching procedure wherein a set of geo-referenced feature descriptors matching a plurality of feature descriptors determined for the at least one image are identified; and

determining three coordinates of a vehicle orientation based on the position information associated with the set of geo-referenced feature descriptors, the three coordinates of the vehicle orientation defining three angles.

18. The method of claim 17, further comprising:

controlling an optical output device based on the three determined coordinates of the vehicle orientation when an augmented reality mode is activated.