Acquisition of three-dimensional images by an active stereo technique using locally unique patterns

• US 20040105580A1
• Filed: 11/07/2003
• Published: 06/03/2004
• Est. Priority Date: 11/22/2002
• Status: Active Grant

• Current Assignee: Strider Labs, Inc.
• Original Assignee: Strider Labs, Inc.
• Inventors: Hager, Gregory D., Wegbreit, Eliot Leonard
• Granted Patent: US 7,103,212 B2
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: G01B 11/2545 with one projection direction and several detection directions, e.g. stereo
• CPC Class Codes: G06T 7/596 from three or more stereo images
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image

Abstract

A system and method for acquiring three-dimensional (3-D) images of a scene. The system includes a projection device for projecting a locally unique pattern (LUP) onto a scene, and sensors for imaging the scene containing the LUP at two or more viewpoints. A computing device matches corresponding pixels in the images by using the local uniqueness of the pattern to produce a disparity map. A range map can then be generated by triangulating points in the imaged scene.

Claims

1. A system for obtaining a disparity map of a scene, comprising:

(a) at least one projection device configured to project a locally unique pattern onto at least a portion of the scene;

(b) sensors for acquiring a plurality of images of the scene, each one of the plurality of images being associated with a unique viewing position; and

(c) a computing device configured to match corresponding pixels in at least two of said images containing said locally unique pattern to generate said disparity map.

2. The system of claim 1, wherein the computer device is additionally configured to compute the range to points in the scene using the disparity map.

3. The system of claim 1, wherein said locally unique pattern is constructed from pattern values consisting of binary values, in which each pattern value is black or white.

4. The system of claim 1 wherein said locally unique pattern is constructed from pattern values consisting of gray-level values, in which each pattern value has a intensity chosen from a plurality of possible intensity values.

5. The system of claim 1, wherein said locally unique pattern is constructed from pattern values consisting of color values, in which each pattern value has three intensity values, one for each of red, green, and blue.

6. The system of claim 1, wherein said locally unique pattern is constructed from pattern values consisting of multi-spectral values, in which each pattern value has intensity values in a plurality of spectral bands, said spectral bands including at least one of ultraviolet light, visible light, and infrared light.

7. The system of claim 1, wherein said locally unique pattern is constructed from pattern values consisting of temporally coded values, in which each pattern value consists of a plurality of temporal bands, each temporal band being projected at a discrete time.

8. The system of claim 1, wherein said locally unique pattern is visually imperceptible to a human observer.

9. The system of claim 1, wherein said locally unique pattern includes a set of bands, each of said bands corresponding to at least one continuous function.

10. The system of claim 9, wherein said at least one continuous function is a periodic function.

11. The system of claim 10, wherein said periodic function is a sine function.

12. The system of claim 9, wherein said set of bands includes a plurality of bands each corresponding to a periodic function, and wherein at least two of said plurality of bands have different phases or periods.

13. The system of claim 1, wherein said locally unique pattern is constructed from replicating a base pattern.

14. The system of claim 13, wherein said base pattern is replicated in successive horizontal or vertical lines.

15. The system of claim 13, wherein said base pattern is shifted by one or more pixels in successive vertical or horizontal lines.

16. The system of claim 13, wherein said base pattern is constructed from first, second and third sine waves having respective frequencies of 4.2, 9.5 and 10.2 pixels and respective phase shifts of 0, 1, and 5.5 pixels, sampled to produce a base pattern of length 140.

17. The system of claim 1, wherein said locally unique pattern is a sequence constructed from a discrete alphabet of symbols.

18. The system of claim 17, wherein said locally unique pattern includes a plurality of pattern windows of specified size, and said locally unique pattern has the property that each pattern window differs from every other pattern window by at least a specified Hamming distance.

19. The system of claim 18, wherein said locally unique pattern has the property that every 2 row by 8 column pattern window differs from every other 2 row by 8 column pattern window within any given 2 row by 146 column window that contains it by a Hamming distance of at least 3.

20. The system of claim 18, wherein said locally unique pattern has the property that every 9 row by 2 column pattern window differs from every other 9 row by 2 column pattern window within any given 9 row by 146 column window that contains it by a Hamming distance of at least 3.

21. The system of claim 17, wherein said locally unique pattern has the property that each pattern window is distinguishable from each other pattern window in accordance with a simulated imaging scoring method, said method including the steps of (a) deriving a lowest value of a first match score representative of matching a pattern window with itself, taking into account a pixel offset of one-half, and (b) comparing the lowest value of the first match score to the highest value of a second match score representative of matching the pattern window to other pattern windows.

22. The system of claim 1, wherein said locally unique pattern is chosen to optimize a pattern score that is a function of at least one of:

(a) pattern scaling and foreshortening effects;

(b) discrete sampling effects;

(c) properties of at least one projection device;

(d) properties of at least one sensor; and

(e) properties of the scene.

23. The system of claim 1, wherein said computing device is configured to compare match windows, said match windows being chosen to be sufficiently large as to ensure that said match windows include the image of a locally unique pattern.

24. The system of claim 1, wherein said computing device is configured to compare match windows, said match windows being smaller than that required to include the image of a locally unique pattern.

25. The system of claim 1 wherein the computing device is configured to:

(a) compare a first set of match regions having a first size and shape;

(b) compare a second set of match regions having a second size and shape different from the first size and shape; and

(c) combine the results of steps (a) and (b);

wherein each match region covers a locally non-repeating pattern window.

26. The system of claim 25, wherein said first set of match regions are in the form of horizontally-oriented rectangles and the second set of match regions are in the form of vertically-oriented rectangles.

28. The system of claim 1, wherein said sensors acquire at least three images, each image being associated with a unique viewing position.

29. The system of claim 28, wherein said disparity map is computed simultaneously from all of said at least three images.

30. The system of claim 28, wherein a plurality of pairwise disparity maps are computed, each one of the plurality of pairwise disparity maps being computed from a unique pair of images, and said plurality of pairwise disparity maps are combined to form a composite disparity map.

31. The system of claim 30, wherein a single locally unique pattern is projected.

32. The system of claim 30, wherein at least two locally unique patterns are projected, each locally unique pattern being chosen to optimize the pairwise disparity map computed for a pair of images.

33. The system of claim 1, wherein said at least one projection device is co-located with one of the viewing positions.

34. The system of claim 1, wherein said locally unique pattern is computed online or selected from a set of candidate patterns based on properties of at least one previously constructed disparity map computed on images acquired using at least one other locally unique candidate pattern.

27. The system of claim I, wherein said viewing positions include a first and a second viewing position, said first viewing position being arranged above said second viewing position, and said projection device disposed along the line defined by said first and second viewing positions.

35. A method for obtaining a disparity map of a scene, comprising the steps of:

(a) causing a locally unique pattern to be projected onto at least a portion of the scene;

(b) receiving a plurality of images acquired of the scene, each one of the plurality of images being associated with a unique viewing position; and

(c) matching corresponding pixels in at least two of said images containing said locally unique pattern to generate said disparity map.

36. The method of claim 35, further comprising a step of computing the range to points in the scene using the disparity map.

37. The method of claim 35, wherein said locally unique pattern is constructed from pattern values consisting of gray-level values, in which each pattern value has a intensity chosen from a plurality of possible intensity values.

38. The method of claim 35, wherein said locally unique pattern includes a set of bands, each of said bands corresponding to at least one continuous function.

39. The method of claim 38, wherein said set of bands includes a plurality of bands each corresponding to a periodic function, and wherein at least two of said plurality of bands have different phases or periods,.

40. The method of claim 35, wherein said locally unique pattern is a sequence constructed from a discrete alphabet of symbols.

41. The method of claim 40, wherein said locally unique pattern includes a plurality of pattern windows of specified size, and said locally unique pattern has the property that each pattern window differs from every other pattern window by at least a specified Hamming distance.

42. The method of claim 40, wherein said locally unique pattern has the property that each pattern window is distinguishable from each other pattern window in accordance with a simulated imaging scoring method, said method including the steps of (a) deriving a lowest value of a first match score representative of matching a pattern window with itself, taking into account a pixel offset of one-half, and (b) comparing the lowest value of the first match score to the highest value of a second match score representative of matching the pattern window to other pattern windows.

43. The method of claim 35, wherein said locally unique pattern is chosen to optimize a pattern score that is a function of at least one of:

(a) pattern scaling and foreshortening effects;

(b) discrete sampling effects;

(c) properties of at least one projection device;

(d) properties of at least one sensor; and

(e) properties of the scene.

44. The method of claim 35, wherein the step of matching corresponding pixels includes a step of comparing match windows, said match windows being chosen to be sufficiently large as to ensure that said match windows include the image of a locally unique pattern.

45. The method of claim 35, wherein the step of matching corresponding pixels includes a step of comparing match windows, said match windows being smaller than that required to include the image of a locally unique pattern.

46. The method of claim 35, wherein the step of matching corresponding pixels includes the steps of:

(d) comparing a first set of match regions having a first size and shape;

(e) comparing a second set of match regions having a second size and shape different from the first size and shape; and

(f) combining the results of steps (d) and (e);

and wherein each match region covers a locally non-repeating pattern window.

47. The method of claim 35, wherein the step of comparing pixels includes computing a plurality of pairwise disparity maps, each one of the plurality of disparity maps being computed from a unique pair of images, and combining the plurality of pairwise disparity maps to generate a composite disparity map.

48. A computer-readable medium storing information for performing the steps of:

(d) causing a projection device to project a locally unique pattern onto at least a portion of a scene;

(e) receiving a plurality of images of the scene, each one of the plurality of images being associated with a unique viewing position; and

(f) matching corresponding pixels in at least two of said images containing said locally unique pattern to generate said disparity map.