Camera based position recognition for a road vehicle

• US 20050002558A1
• Filed: 04/29/2004
• Published: 01/06/2005
• Est. Priority Date: 05/23/2003
• Status: Abandoned Application

• Current Assignee: Daimler Chrysler Company LLC (Stellantis NV)
• Original Assignee: Daimler Chrysler Company LLC (Stellantis NV)
• Inventors: Hahn, Stefan, Franke, Uwe
• Application Number:
  US10/835,130
• Publication Number:
  US 20050002558A1
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G06V 2201/08 Detecting or categorising vehicles
• CPC Class Codes: G08G 1/123 indicating the position of vehicles, e.g. scheduled vehicles; Managing passenger vehicles circulating according to a fixed timetable, e.g. buses, trains, tramstransmission of navigation instructions to vehicles G08G1/0968

Abstract

A camera based position recognition system for a road vehicle. The environment in the direction of travel of the vehicle is acquired by a camera. Using the acquired image data, the position of the vehicle in its environment is determined with regard to an optical signature identified in the obtained image data. For determining the position of the vehicle, use is made of the knowledge of the relationship between the environment coordinate system of the optical signature and that of the camera coordinate system. In simplified manner, position determination occurs with regard to the optical signature on the basis of a template matching imposed on the image data. For this, a template or an optical signature recorded in memory is superimposed on the optical signature identified in the image data in the environment of the vehicle (template matching). From the parameters of this template matching (for example linear compression and rotation parameters) recognizing the existing coordinate system, the position of the vehicle relative to the optical signature can be directly deduced.

Claims

1-23. -23. (Cancelled)

24. A process for camera-based position recognition for a road vehicle, comprising:

obtaining image data of the environment in the direction of travel of the vehicle using a camera,

determining the position of the vehicle in its environment relative to an optical signature identified in the obtained image data on the basis of template matching imposed on the image data, and on the basis of knowledge of the relationship between the environment coordinate system and the camera coordinate system.

25. A process according to claim 24, further comprising initiating an at least semi-autonomous vehicle guidance upon determining the position of the vehicle relative to the optical signature, during which the vehicle is brought to a halt at a predetermined position relative to the optical signature.

26. A process according to claim 24, wherein the template matching occurs in a three dimensional coordinate space.

27. A process according to claim 24, wherein the template matching occurs on the basis of an edge image.

28. A process according to claim 24, wherein the template is stored as a list of points, and wherein the template matching occurs on a point-to-point basis.

29. A process according to claim 24, further comprising selecting a template from a number of different templates prior to the matching of the template with the image data.

30. A process according to claim 29, wherein the selection occurs on the basis of GPS or map information.

31. A process according to claim 24, comprising, in the framework of the template matching, in which the camera coordinates are calculated on the basis of the relationship between the image parameters of the template matched to the image data of the optical signature and the coordinates of the vehicle environment,

computationally stepwise changing the orientation of the camera, and

comparing, for each change, the quality of the correspondence or fit of the depiction of the template with the image data of the optical signature.

32. A process according to claim 31, wherein for calculation of the quality of the correspondence a standard correlation is calculated according to

$c$

( s , t ) = ∑

x ⁢

∑

y ⁢

f ⁡

( x , y ) ⁢

w ⁡

( x + s , y + t )

wherein the summation occurs over those ranges, for which the depiction of the template w(x,y) and the image data of the optical signature f(x,y) overlap, and

wherein the maximal value of c(s,t) then occurs, when the correspondence of the depiction of the template has the best fit with the image data of the optical signature.

33. A process according to claim 31, comprising

subjecting the image data to a distance transformation prior to the computation of the standard correlation, and

subsequently calculating the standard correlation of the distance transformed image with the depiction of the template according to

$\mathrm{DS}=\sum _{}^{}$

i=1n⁢

Dim⁡

(T⁡

[template])=f⁡

(α

,β

,φ

,X,Y,Z)

wherein α

, β

, φ

, X, Y, Z represent the pitch, tilt and roll angles of the vehicle, as well as the camera pose in the longitudinal, lateral orientation and in its height, T describes the transformation of the coordinate system of the environment into the camera coordinates and D_m represents the value the distance transformation of those image data, which correspond with the appropriate points in the template, and wherein DS then is minimal, when the correspondence of the depiction of the template exhibits the highest wellness of fit with the image data of the optical signature.

34. A process according to claim 33, further comprising making use of Powell minimization for determining the actual minimum DS.

35. A process according to claim 33, further comprising improving the estimation of the actual minimum by the subsequent use of a Kalman Filter.

45. A process according to claim 24, wherein said position recognition and/or a step of destination guidance is carried out in road vehicles in the vicinity of bus stops.

46. A process according to claim 24, wherein said position recognition and/or a step of destination guidance is carried out in road vehicles in the vicinity of parking spaces or parking lots or parking garages.

36. A device for camera based position recognition for a road vehicle, comprising:

a camera for detecting the environment in the direction of travel of the vehicle, and

an image processing unit with object recognition for determining the position of the vehicle with regard to an optical signature in the environment of the vehicle,

a memory unit in which a template is stored corresponding to the optical signature, the memory unit in communication with the image processing unit for position determination, and

a means for template matching accessible to the image processing unit, via which means the template can be superimposed over the image data stored in the memory.

37. The device according to claim 36, wherein the template stored in the memory unit is a three dimensional template.

38. The device according to claim 36, wherein the template substantially corresponds to the edge image of the optical signature.

39. The device according to claim 36, wherein the template is organized as a list of points, such that the template matching occurs on a point-to-point basis.

40. The device according to claim 36, wherein the templates of a number of various optical signatures are stored in the memory unit, which can be selected from for template matching.

41. The device according to claim 40, wherein the device is in communication with a unit for position determination, in particular a GPS system or a map navigation system, via which the selection of the templates for template matching is controlled.

42. The device according to claim 36, wherein the image processing unit includes means via which the image data obtained by the camera can be subjected to a distance transformation, and means via which the standard correlation of the distance transformed image with the depiction of the template of the optical signature is calculated according to

$\mathrm{DS}=\sum _{}^{}$

i = 1 n ⁢

D im ⁡

( T ⁡

[ template ] ) = f ⁡

( α

, β

, φ

, X , Y , Z )

43. The device according to claim 42, wherein the image processing unit includes a device for Powell minimization for determining the actual minimum DS.

44. The device according to claim 42, wherein the image processing unit includes a Kalman Filter adapted to further improve the estimation of the actual minimum.