Creation of a Range Image

• US 20090066929A1
• Filed: 11/03/2006
• Published: 03/12/2009
• Est. Priority Date: 01/08/2006
• Status: Active Grant

• Current Assignee: Visiontools Bildanalyse Systeme GmbH
• Original Assignee: Hermann Tropf
• Inventors: Tropf, Hermann
• Granted Patent: US 7,804,585 B2
• Time in Patent Office: Days
• US Class Current: 356/3.120
• CPC Class Codes: G01B 11/2513 with several lines being projected in more than one direction, e.g. grids, patterns
• CPC Class Codes: G01B 11/2545 with one projection direction and several detection directions, e.g. stereo
• CPC Class Codes: G01C 11/06 by comparison of two or more pictures of the same area
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light

Abstract

The most difficult problem in the creation of a range image with stereo cameras (4, 5) is the establishing of the correspondence of the points. For this, the scene (3) is illuminated twice; thereof at least once with a random or pseudo random pattern. For both cameras (4, 5), an image is taken for each of the illuminations and the quotient of brightnesses is calculated pixelwise. The correspondence is established on the basis of a comparison of the quotient of pixels on epipolar lines of different cameras (4, 5). The illumination pattern is preferably highly modulated along the epipolar line; transversally or diagonally to it, it is not or only slightly modulated.

For illumination, a projection unit (9) is used which in a preferred arrangement comprises two superposed grating patterns (21, 22) which have a distance (d) to each other, with at least one varying in a pseudo random manner, and with two closely neighboring light sources (25, 26) which shine through the gratings (21, 22) and thereby generate different pseudo random patterns, especially moiré patterns.

Applications are found, inter alia, in assembly control, robotics, metrology, archaeology, in the textile industry, in biometry, medicine, reverse engineering.

Claims

1. Method for the acquisition of a range image of a scene from the correspondence of pixels of the images of a first camera (4) and at least one second camera (5) which record the scene in stereo arrangement,

characterized in that

the scene is illuminated in a first illumination with a random or pseudo random pattern which is designed, with regard to at least one modulation feature in at least one spatial dimension, as a random sequence or, respectively, pseudo random sequence;

in the first illumination, a first image is taken by at least two cameras (4, 5);

in a second illumination, the scene is illuminated, wherein a second image with the second illumination is taken by the at least two cameras (4, 5);

for at least two cameras (4, 5), a ratio is calculated pixel-wise from the first and the second image taken by the corresponding camera (4, 5), in which a modulation feature value of the pixel from the one image is placed in the numerator, and the modulation feature from the other image is placed in the denominator of the ratio value; and

the correspondence of pixels of the scene is determined on the basis of a comparison of the modulation feature ratios of pixels from different cameras (4,5).

2. Method according to claim 1, characterized in that the modulation feature is the brightness or, respectively, the intensity of the illumination and the modulation feature ratio is a brightness or, respectively, intensity ratio.

3. Method according to claim 1, characterized in that the modulation feature is a color parameter of the illumination and the modulation feature ratio is a color value ratio.

4. Method according to any one of the preceding claims, characterized in that the random or, respectively, pseudo random sequence is a binary sequence, i.e. a sequence with two conditions of the modulated feature, preferably a sequence of light/dark values.

5. Method according to any one of the preceding claims, characterized in that—

in forming the modulation feature ratio—

an offset value is added to or subtracted from the modulation features set into a ratio.

6. Method according to any one of the preceding claims, characterized in that the random or, respectively, pseudo random pattern is essentially locally modulated in a first spatial dimension and essentially not modulated or only slightly in the second spatial dimension.

7. Method according to any one of the preceding claims, characterized in that—

with the second illumination—

a homogeneous illumination, a random or pseudo random pattern similar to the first illumination, or, preferably, a random or pseudo random pattern different from the first illumination are used.

8. Method according to any one of the preceding claims, characterized in that the first and the second illumination taking the first and the second image are performed at least approximately from the same solid angle.

9. Method according to any one of the preceding claims, characterized in that the modulation feature ratios are compared on epipolar lines allocated to each other from different cameras (4, 5).

10. Method according to the preceding claim, characterized in that correspondence is obtained by a comparison of the values of a piecewise, one-dimensional, standardized correlation of the modulation feature ratios along the epipolar lines.

11. Method according to the claims 6 and 9, characterized in that the random or, respectively, pseudo random pattern is essentially modulated along the epipolar lines.

12. Method according to claim 11, characterized in that the second spatial dimension of the random or, respectively, pseudo random pattern is oriented transversely to the epipolar lines.

13. Method according to claim 11, characterized in that the second spatial dimension of the random or, respectively, pseudo random pattern is oriented diagonally to the epipolar lines.

14. Method according to any one of the claims 9 to 13, especially according to claim 13, characterized in that—

in the analysis of the image information for the creation of the range image—

allocations of values different from each other are included which belong to the same or nearly the same object point but come from different epipolar lines which are neighboring or arranged closely to each other.

15. Method according to any one of the preceding claims, characterized in that the first and the second illumination are provided with different colors, preferably simultaneously, and that color cameras are used.

16. Method according to any one of the preceding claims, characterized in that the first and the second illumination are provided with a different polarization, and in that the channels, upon taking the image, are separated by beam splitters and downstream polarization filters.

37. Method according to claim 36, characterized in that it comprises one or a plurality of features of a method according to any one of the claims 1 to 16.

40. Application of a method or a device according to any one of the preceding claims in the field of assembly control, robotics, metrology, archaeology, textile industry, biometry, medicine or reverse engineering.

17. Device for the acquisition of a range image of a scene comprising at least two cameras (4, 5) in stereo arrangement, at least one image analysis unit for analyzing the camera pictures and at least one projection unit (9) for illuminating the scene,

characterized in that

by means of the projection unit (9), at least two different illuminations can be projected onto the scene, of which at least one illumination is a random or pseudo random pattern which is designed, with regard to at least one modulation feature in at least one spatial dimension, as a random sequence or, respectively, pseudo random sequence;

in the first illumination, a first image can be taken by at least two cameras (4, 5);

in a second illumination, the scene can be illuminated wherein a second image with the second illumination can be taken by the at least two cameras (4, 5);

for at least two cameras (4, 5), a ratio can be calculated pixel-wise with the image analysis unit from the first and the second image taken by the corresponding camera (4, 5), in which a modulation feature value of the pixel from the one image is placed in the numerator, and the modulation feature from the other image is placed in the denominator of the ratio value; and

the correspondence of pixels of the scene can be determined on the basis of a comparison of the modulation feature ratios of pixels from different cameras (4, 5).

18. Device according to the preceding claim, characterized in that at least two illuminations can be performed with a random or pseudo random pattern.

19. Device according to any one of the claims 17 to 18, characterized in that at least one random or pseudo random pattern is a brightness pattern.

20. Device according to any one of the claims 17 to 19, characterized by a projection unit (9) with two superposed gratings (21, 22) which have a distance (d) to each other, wherein at least one has a random or pseudo random structure, and by an illumination with at least two light sources (25, 26) which shine through the gratings (21, 22) or can illuminate them and thereby provide different patterns.

21. Device according to any one of the claims 17 to 19, characterized by a projection unit (9) with two superposed gratings (15, 16, 17, 18, 19, 20) which have a distance to each other, wherein at least for one, the phase and/or the frequency of the grating varies in a pseudo random manner, and by an illumination with at least two light sources which shine through the gratings or can illuminate them and thereby provide different moiré

patterns.

22. Device according to any one of the claims 17 to 19, characterized by a projection unit (9) with a transparent or partly transparent layer in the optical path of the projection unit (9), comprising a random or pseudo random structure, and by an illumination with at least two light sources which shine through the structure or can illuminate it and thereby provide different patterns.

23. Device according to the preceding claim, characterized in that the transparent or partly transparent layer is a body with a slight surface waviness or surface roughness, preferably a thin wavy pane of glass (35).

24. Device according to any one of the claims 17 to 19, characterized by a locally differently transparent mask in the form of a pseudo random pattern which can be changed through electrical control.

25. Device according to any one of the claims 17 to 19, characterized by one or a plurality of projection units (9), each comprising at least two closely neighboring, approximately point or line shaped light sources whose cones of light emit different brightness patterns.

26. Device according to any one of the claims 17 to 19, characterized by a projection unit (9) with two superposed patterns movable to each other, with at least one having a pseudo random structure.

27. Device according to any one of the claims 17 to 19, characterized by a projection unit (9) with two superposed patterns movable to each other, with at least one in which the phase and/or the frequency varies in a pseudo random manner.

28. Device according to any one of the claims 17 to 19, characterized by a projection unit (9) comprising a random or pseudo random structure which can be moved rotatorily or translatorily.

29. Device according to any one of the claims 17 to 19, characterized by a projection unit (9) with a mask which includes a transparent vessel (32) in which stochastically partially transparent or opaque particles (31) are moving.

30. Device according to any one of the claims 17 to 19, characterized by a projection unit (9) with at least one approximately point- or line-shaped light source whose cones of light emit brightness patterns with constant pseudo random patterns, with the light sources being movable by vibration.

31. Device according to any one of the claims 17 to 19, characterized by a slot of light moving over the scene, the brightness of this slot being modulated in a random or pseudo random manner.

32. Device according to any one of the claims 17 to 31, characterized in that a random or, respectively, pseudo random pattern is projectable on the scene, which is essentially locally modulated in a first spatial dimension and essentially not or only slightly modulated in the second spatial dimension.

33. Device according to claim 32, characterized in that the random or, respectively, pseudo random pattern is essentially modulated along the epipolar lines.

34. Device according to any one of the claims 32 to 33, characterized in that the second spatial dimension of the random or, respectively, pseudo random pattern is transversely aligned to the epipolar lines.

35. Device according to any one of the claims 32 to 33, characterized in that the second spatial dimension of the random or, respectively, pseudo random pattern is diagonally aligned to the epipolar lines.

39. Device according to claim 38, characterized in that it comprises one or a plurality of features of a device according to any one of the claims 17 to 36.

36. Method for the creation of a range image of a scene from the correspondence of pixels of the images of a first camera (4) and at least one second camera (5) which record the scene in stereo arrangement,

characterized in that

in a first illumination, the scene is illuminated with a random or pseudo random pattern which is designed, with regard to at least one modulation feature in at least one spatial dimension, as a random sequence or, respectively, pseudo random sequence;

the random or, respectively, pseudo random pattern is essentially locally modulated in the first spatial dimension and essentially not or only slightly modulated in the second spatial dimension;

the second spatial dimension of the random or, respectively, pseudo random pattern is aligned diagonally to the epipolar lines;

in the first illumination, a first image is taken by at least two cameras (4, 5);

the comparison of the modulation features is performed on allocated epipolar lines of different cameras (4, 5); and

the correspondence of pixels of the scene is determined on the basis of a comparison of the modulation features of pixels from different cameras (4, 5).

38. Device for the creation of a range image of a scene, comprising at least two cameras (4, 5) in stereo arrangement, at least one image analysis unit for analyzing the camera pictures and at least one projection unit (9) for illumination of the scene,

characterized in that

by means of the projection unit, a first illumination can be projected onto the scene, designed as a random or pseudo random pattern which is designed, with regard to at least one modulation feature in at least one spatial dimension, as a random sequence or, respectively, pseudo random sequence;

the random or, respectively, pseudo random pattern is essentially locally modulated in the first spatial dimension and essentially not or only slightly modulated in the second spatial dimension;

the second spatial dimension of the random or, respectively, pseudo random pattern is aligned diagonally to the epipolar lines;

in the first illumination, a first image can be taken by at least two cameras (4, 5); and

the correspondence of pixels of the scene is determined on the basis of a comparison of the modulation features of pixels from different cameras (4, 5).