3D geometric modeling and motion capture using both single and dual imaging

• US 8,090,194 B2
• Filed: 08/13/2007
• Issued: 01/03/2012
• Est. Priority Date: 11/21/2006
• Status: Active Grant

• Current Assignee: Mantis Vision, Ltd.
• Original Assignee: Mantis Vision, Ltd.
• Inventors: Bittan, Gur Arie, Golrdon, Eyal
• Primary Examiner(s): WERNER, BRIAN P
• Application Number:
  US11/837,553
• Publication Number:
  US 20080118143A1
• Time in Patent Office: 1,604 Days
• Field of Search: None
• US Class Current: 382/154
• CPC Class Codes: G01B 11/2513 with several lines being projected in more than one direction, e.g. grids, patterns
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06V 10/145 Illumination specially adapted for pattern recognition, e.g. using gratings
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19

Abstract

A method and apparatus for obtaining an image to determine a three dimensional shape of a stationary or moving object using a bi dimensional coded light pattern having a plurality of distinct identifiable feature types. The coded light pattern is projected on the object such that each of the identifiable feature types appears at most once on predefined sections of distinguishable epipolar lines. An image of the object is captured and the reflected feature types are extracted along with their location on known epipolar lines in the captured image. Displacements of the reflected feature types along their epipolar lines from reference coordinates thereupon determine corresponding three dimensional coordinates in space and thus a 3D mapping or model of the shape of the object at any point in time.

Claims

1. A method of obtaining distance data from a 2D (two-dimensional) image of a scene comprising one or more objects, said method comprising:

projecting on the objects a coded light pattern comprising multiple appearances of a finite set of feature types, each feature type being distinguishable according to a unique bi-dimensional formation, such that a distance between adjacent epipolar lines associated with substantially identical appearances of any given feature type is minimized according to a limiting epipolar separation factor, giving rise to a plurality of distinguishable epipolar lines separated by approximately a minimum safe distance for epipolar line distinction;

capturing a 2D image of the objects having the projected coded light pattern projected thereupon; and

extracting reflected feature types according to the unique bi-dimensional formations and locations of the reflected feature types on respective epipolar lines in the 2D image.

2. The method according to claim 1, wherein the number of distinguishable epipolar lines is determined by minimizing the distance between the adjacent epipolar lines associated with substantially identical appearances of a given feature type to a point where the adjacent epipolar lines are no longer distinguishable.

3. The method according to claim 1, wherein the distance between the adjacent epipolar lines associated with substantially identical appearances of a given feature type is minimized according to the limiting epipolar separation factor and according to a safety factor.

4. The method according to claim 3, wherein the distance between the adjacent epipolar lines associated with the substantially identical appearances of the given feature type is minimized according to the limiting epipolar separation factor to a point, where according to the safety factor, the distance between the adjacent epipolar lines is too small to provide a safe distance for epipolar line distinction.

5. The method according to claim 1, wherein the limiting epipolar separation factor is associated with one or more of the following:

optical characteristics of a imaging apparatus, optical characteristics of a projector, coordinate detection algorithms, feature type characteristics, object surface type and projector SNR level.

6. The method according to claim 3, wherein the safety factor is associated with one or more of the following:

a parameter representing mechanical stability of a calibrated projector, a parameter representing mechanical stability of a calibrated imaging apparatus and a parameter representing encoded pattern directionality.

7. The method according to claim 1, wherein said projecting further comprises projecting the coded light pattern on the objects at an angle in relation to the epipolar lines, the angle corresponds to the minimum distance determined in accordance with the limiting epipolar separation factor.

8. The method according to claim 7, further comprising orientating or angling or tilting or skewing or rotating the coded light pattern in relation to the epipolar lines to enable said projecting the coded light pattern on the objects at an angle in relation to the epipolar lines.

9. The method according to claim 7, wherein said orientating or angling or tilting or skewing or rotating the coded light pattern comprises providing a non-periodic pattern, and wherein the non-periodic pattern is arranged approximately to provide an angle in relation to the epipolar lines, is associated to the minimum safe distance for epipolar line distinction.

10. The method according to claim 1, wherein said projecting comprises projecting the coded light pattern such that the given feature type appears at most once on two or more predefined sections of the distinguishable epipolar lines, each section representing a corresponding depth range.

11. The method according to claim 1, wherein said projecting comprises projecting the coded light pattern at an angle in relation to the epipolar lines in accordance with the parameters of a Sin−

1 (P/YC) formula.

12. The method according to claim 1, wherein said extracting comprises determining elements that comprise the reflected feature types, and extracting epipolar distances between the elements of the reflected feature types.

13. The method according to claim 12, wherein the epipolar distances between the elements that comprise the reflected feature types are varying distances.

14. The method according to claim 1, wherein the epipolar lines are characterized by varying x and/or y coordinates.

15. The method according to claim 1, wherein the epipolar lines are curved.

16. The method according to claim 1, wherein the 2D image is a video frame.

17. An apparatus configured to obtain distance data from a 2D (two-dimensional) image of a scene comprising one or more objects, said apparatus comprising:

a bi-dimensional coded light pattern comprising multiple appearances of a finite set of feature types, each feature type being distinguishable according to a unique bi-dimensional formation; and

a projector configured to project the coded light pattern on the objects, such that a distance between adjacent epipolar lines associated with substantially identical appearances of any given feature type is minimized according to a limiting epipolar separation factor, giving rise to a plurality of distinguishable epipolar lines separated by approximately a minimum safe distance for epipolar line distinction;

wherein the projector enables an imaging unit to capture a 2D image of the objects having the projected coded light pattern projected thereupon to enable an image processing unit to extract reflected feature types according to the unique bi-dimensional formations, and to determine locations of the reflected feature types on respective epipolar lines in the 2D image.

18. The apparatus according to claim 17, wherein said projector is configured to project the coded light pattern on the objects, such that the distance between adjacent epipolar lines associated with substantially identical appearances of any given feature type is minimized to a point where the adjacent epipolar lines are no longer distinguishable.

19. The apparatus according to claim 17, wherein the projector is configured to project the coded light pattern on the objects, such that the distance between adjacent epipolar lines associated with substantially identical appearances of a given feature type is minimized according to the limiting epipolar separation factor and according to a safety factor.

20. The apparatus according to claim 19, wherein the projector is configured to project the coded light pattern on the objects, such that the distance between the adjacent epipolar lines associated with substantially identical appearances of a given feature type is minimized according to the limiting epipolar separation factor to a point, where according to the safety factor, the distance between the adjacent epipolar lines is too small to provide a safe distance for epipolar line distinction.

21. The apparatus according to claim 17, wherein projector is configured to project the coded light pattern on the objects according to the limiting epipolar separation factor which corresponds to one or more of the following:

optical characteristics of a imaging apparatus, optical characteristics of a projector, coordinate detection algorithms, feature type characteristics, object surface type and projector SNR level.

22. The apparatus according to claim 19, wherein said projector is further configured to project the coded light pattern on the objects according to the safety factor which corresponds to one or more of the following:

a parameter representing mechanical stability of a calibrated projector, a parameter representing mechanical stability of a calibrated imaging apparatus and a parameter representing encoded pattern directionality.

23. The apparatus according to claim 17, wherein said projector is configured to project the coded light pattern on the objects at an angle in relation to the epipolar lines, the angle is associated with the limiting epipolar separation factor.

24. The apparatus according to claim 23, wherein said projector is configured to orientate or angle or tilt or skew or rotate the coded light pattern in relation to the epipolar lines according to the limiting epipolar separation factor.

25. The apparatus according to claim 23, wherein the coded light pattern projected by said projector is a non-periodic pattern, and wherein the non-periodic pattern is arranged to approximately provide an angle in relation to the epipolar lines, the angle is associated to the minimum safe distance for epipolar line distinction.

26. The apparatus according to claim 17, wherein said projector is adapted to project the coded light pattern such that the given feature type appears at most once on two or more predefined sections of the distinguishable epipolar lines, each section representing a corresponding depth range.

27. The apparatus according to claim 17, wherein the projector is configured such that the projecting of the coded light pattern is at an angle in relation to the epipolar lines in accordance with the parameters of a Sin−

1 (P/YC) equation

28. The apparatus according to claim 17, wherein the projector enables a processing unit to determine elements that comprise the reflected feature types, and to determine epipolar distances between the elements of the reflected feature types.

29. The apparatus according to claim 28, wherein the epipolar distances between the elements that comprise said reflected feature types are varying distances.

30. The apparatus according to claim 17, wherein the epipolar lines are characterized by varying x and/or y coordinates.

31. The apparatus according to claim 17, wherein the epipolar lines are curved.

32. The apparatus according to claim 17, wherein the 2D image is a video frame.