Apparatus, method and computer program for determining a position on the basis of a camera image from a camera

• US 8,121,350 B2
• Filed: 12/27/2007
• Issued: 02/21/2012
• Est. Priority Date: 12/29/2006
• Status: Active Grant

• Current Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Original Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Inventors: Klefenz, Franz
• Primary Examiner(s): Ahmed, Samir
• Assistant Examiner(s): Rashidian, Mehdi
• Application Number:
  US12/521,359
• Publication Number:
  US 20100329513A1
• Time in Patent Office: 1,517 Days
• Field of Search: 382/100, 382103-104, 382106-107, 382/195, 382/190, 348/113, 348148-149, 348/222.1, 345/173, 345441-443, 715/863, 455/456.1, 340/907, 340/933, 701 1- 7, 701/207
• US Class Current: 382/104
• CPC Class Codes: G06T 2207/20061 Hough transform
• CPC Class Codes: G06T 2207/20164 Salient point detection; Corner detection
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/48 by mapping characteristic values of the pattern into a parameter space, e.g. Hough transformation
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

An apparatus for determining a position on the basis of a camera image from a camera includes a Hough transformer, a positional description establisher and a database comparator. The Hough transformer is formed to identify circular arcs or elliptical arcs in the camera image or in a preprocessed version of the camera image derived therefrom, and to identify a plurality of straight stretches passing in various directions through the camera image or through the preprocessed version. The positional description establisher is formed to obtain a positional description describing the identified circular arcs or elliptical arcs and the identified straight stretches by parameters, on the basis of the identified circular arcs or elliptical arcs and on the identified straight stretches. The database comparator further is formed to compare the positional description with a plurality of comparative positional descriptions and to obtain information on a position as a result of the comparison.

Claims

1. An apparatus for determining a position on the basis of a camera image from a camera, comprising:

a Hough transformer formed to identify circular arcs or elliptical arcs in the camera image or in a preprocessed version of the camera image derived therefrom, and

to identify a plurality of straight stretches passing in various directions through the camera image or through the preprocessed version,

wherein the Hough transformer is formed to provide information on a location of an identified circular arc or elliptical arc, and to provide information on a radius of curvature of the identified circular arc or elliptical arc;

a positional description establisher formed to acquire a positional description describing the identified circular arcs or elliptical arcs and the identified straight stretches by parameters, on the basis of the identified circular arcs or elliptical arcs and on the identified straight stretches,

wherein the positional description for identified circular arcs or elliptical arcs comprises information on a location of an identified circular arc or elliptical arc and information on a radius of curvature of the identified circular arc or elliptical arc; and

a database comparator formed to compare the positional description with a plurality of comparative positional descriptions, and

to acquire information on a position as a result of the comparison,

wherein the database comparator is formed to form a difference between a feature vector of the positional description and feature vectors of the comparative positional descriptions,

and to output, as the result, positional information belonging to the corresponding comparative positional description in response to the difference for one of the comparative positional descriptions being sufficiently small.

2. The apparatus according to claim 1, wherein the positional description establisher is formed to acquire a description of a course of the road, which reproduces the stretch course as an ordered description of identified circular arcs or elliptical arcs, as the positional description.

3. The apparatus according to claim 1, wherein the positional description establisher is formed to acquire the positional description so that the positional description comprises a description of a roadway edge, a lane boundary or a roadway center.

4. The apparatus according to claim 1, wherein the Hough transformer is formed to provide information on a location, an arc length, a radius of curvature, or a radius of curvature of the identified circular arcs or elliptical arcs, and provide information on a location, a length or a direction of the identified straight stretches.

5. The apparatus according to claim 1, wherein the positional description establisher is formed to acquire the positional description from arc course parameters and straight stretch course parameters provided by the Hough transformer, which describe the identified circular arcs or elliptical arcs and the identified straight stretches.

6. The apparatus according to claim 1, wherein the positional description establisher is formed to establish a description of the course of the road by patching selected adjacent identified circular arcs or elliptical arcs or straight stretches together,

wherein the positional description establisher is formed to select the selected circular arcs or elliptical arcs or straight stretches used for the description of the course of the road from an entirety of identified circular arcs or elliptical arcs and identified straight stretches so that the selected circular arcs or elliptical arcs and straight stretches describe a continuous course of the road from a default starting point to a default end point.

7. The apparatus according to claim 1, wherein the apparatus comprises an image mapper, which is formed to map the image from the camera or a preprocessed camera image by projective mapping so that a preprocessed image developing thereby shows a top view onto a landscape captured by the camera when the image from the camera shows a panoramic view of the landscape.

8. The apparatus according to claim 1, wherein the apparatus comprises an image assembler, which is formed to assemble a plurality of camera images or preprocessed camera images captured at various time instants to an assembled image.

9. The apparatus according to claim 8, wherein the image assembler is formed to shift or rotate a first image to be assembled and a second image to be assembled with respect to each other until a deviation of the assembled images rotated with respect to each other or shifted with respect to each other in an overlapping region is sufficiently small in accordance with a predefined deviation rule.

10. The apparatus according to claim 8, wherein the image assembler is formed to choose a starting value for a shift of the images to be assembled on the basis of a previously determined mean shift vector.

11. The apparatus according to claim 1, wherein the positional description establisher is formed to generate a feature vector describing successive sections of the course of the road on the basis of the identified circular arcs or elliptical arcs and on the identified straight stretches.

12. The apparatus according to claim 11, wherein the database comparator is formed to compare the feature vector with a plurality of comparative feature vectors associated with positions,

to acquire a measure of differences between the feature vector and the comparative feature vectors, and

to determine the position belonging to the feature vector on the basis of the measure of the difference.

13. The apparatus according to claim 1, wherein the apparatus comprises an edge detector, which is formed to detect edges in the camera image, and

to generate the edge image as a preprocessed version of the camera image on the basis of the camera image.

14. The apparatus according to claim 13, wherein the edge detector is formed to convert the camera image to a monochrome image, using a threshold value, and detect the edge in the monochrome image.

15. The apparatus according to claim 1, wherein the apparatus comprises an edge detector, which is formed to detect edges in the image and to generate an edge image on the basis of the image;

wherein the Hough transformer is formed to provide location information on the identified circular arcs or elliptical arcs, and

to provide location information on the identified straight stretches;

wherein the positional description establisher is formed to generate a description of the course of the road in form of an ordered combination of circular arcs or elliptical arcs and of straight stretches, which together describe the course of the road, on the basis of the identified circular arcs or elliptical arcs and the identified straight stretches; and

wherein the database comparator is formed to compare the description of the course of the road in form of the ordered description with a plurality of comparative descriptions, which describe comparative courses of roads in form of ordered combinations of circular arcs or elliptical arcs and of straight stretches,

to determine a measure of deviations between the description of the course of the road and the comparative descriptions, and

to provide the positional information associated with a certain comparative course of the road as the particular position if a deviation between the description of the course of the road and the comparative description of the particular comparative course of the road is smaller than a default threshold or smaller than deviations determined for the remaining comparative courses of roads.

16. The apparatus according to claim 1, wherein the apparatus is formed to determine a position of a vehicle on the basis of the camera image or on the basis of the plurality of camera images.

17. The apparatus according to claim 1, wherein the camera is attached at a vehicle to capture camera images of a landscape located in front of the vehicle or a landscape located behind the vehicle.

18. The apparatus according to claim 1, wherein the positional description establisher is formed to acquire a description of an environment describing the environment as a representation of identified circular arcs or elliptical arcs or straight stretches, as the positional description.

19. The apparatus according to claim 18, wherein the database comparator is formed to compare the positional description with a plurality of comparative positional descriptions,

to acquire the information on the position as a result of the comparison,

wherein the comparative positional descriptions describe circular arcs or elliptical arcs and straight stretches comprised in a three-dimensional model of the environment.

20. A method of determining a position on the basis of a camera image from a camera, comprising:

Hough-transforming the camera image or a preprocessed version of the camera image, in order to identify circular arcs or elliptical arcs in the camera image or in the preprocessed version of the camera image, and to identify a plurality of straight stretches passing in various directions through the camera image or through the preprocessed version derived therefrom,

wherein, by Hough-transforming, information on a location of an identified circular arc or elliptical arc and information on a radius of curvature of the identified circular arc or elliptical arc are provided;

establishing a positional description describing the identified circular arcs or elliptical arcs and the identified straight stretches by parameters, on the basis of the identified circular arcs or elliptical arcs and the identified straight stretches,

wherein the positional description for identified circular arcs or elliptical arcs comprises information on a location of an identified circular arc or elliptical arc and information on a radius of curvature of the identified circular arc or elliptical arc; and

comparing the positional description with a plurality of comparative positional descriptions in a database, in order to acquire information on the position as a result of the comparison,

wherein a difference between a feature vector of the positional description and feature vectors of the comparative positional descriptions is being formed,

and wherein positional information belonging to the corresponding comparative positional description is output as the result in response to the difference for one of the comparative positional descriptions being sufficiently small.

21. A computer readable non-transitory medium having a computer program for performing, when the computer program is executed on a computer, a method of determining a position on the basis of a camera image from a camera, the method comprising:

Hough-transforming the camera image or a preprocessed version of the camera image, in order to identify circular arcs or elliptical arcs in the camera image or in the preprocessed version of the camera image, and to identify a plurality of straight stretches passing in various directions through the camera image or through the preprocessed version derived therefrom,

wherein, by Hough-transforming, information on a location of an identified circular arc or elliptical arc and information on a radius of curvature of the identified circular arc or elliptical arc are provided;

establishing a positional description describing the identified circular arcs or elliptical arcs and the identified straight stretches by parameters, on the basis of the identified circular arcs or elliptical arcs and the identified straight stretches,

wherein the positional description for identified circular arcs or elliptical arcs comprises information on a location of an identified circular arc or elliptical arc and information on a radius of curvature of the identified circular arc or elliptical arc; and

comparing the positional description with a plurality of comparative positional descriptions in a database, in order to acquire information on the position as a result of the comparison,

wherein a difference between a feature vector of the positional description and feature vectors of the comparative positional descriptions is being formed,

and wherein positional information belonging to the corresponding comparative positional description is output as the result in response to the difference for one of the comparative positional descriptions being sufficiently small.