3D geometric modeling and 3D video content creation

• US 8,538,166 B2
• Filed: 11/20/2007
• Issued: 09/17/2013
• Est. Priority Date: 11/21/2006
• Status: Active Grant

• Current Assignee: Mantis Vision, Ltd.
• Original Assignee: Mantis Vision, Ltd.
• Inventors: Gordon, Eyal, Bittan, Gur Arieh
• Primary Examiner(s): Bella, Matthew
• Assistant Examiner(s): Rosario, Dennis
• Application Number:
  US12/515,715
• Publication Number:
  US 20100074532A1
• Time in Patent Office: 2,128 Days
• Field of Search: 382/154, 382/195, 382/201
• US Class Current: 382/201
• CPC Class Codes: G01B 11/25 by projecting a pattern, e.g. one or more lines, moiré fringes on the object G01B11/255 takes precedence ; image analysis for depth or shape recovery G06T7/50
• CPC Class Codes: G01B 11/2513 with several lines being projected in more than one direction, e.g. grids, patterns
• CPC Class Codes: G01C 11/025 by scanning the object
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/20228 Disparity calculation for image-based rendering
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06T 7/536 from perspective effects, e.g. by using vanishing points
• CPC Class Codes: G06V 10/145 Illumination specially adapted for pattern recognition, e.g. using gratings

Abstract

A system, apparatus and method of obtaining data from a 2D image in order to determine the 3D shape of objects appearing in said 2D image, said 2D image having distinguishable epipolar lines, said method comprising: (a) providing a predefined set of types of features, giving rise to feature types, each feature type being distinguishable according to a unique bi-dimensional formation; (b) providing a coded light pattern comprising multiple appearances of said feature types; (c) projecting said coded light pattern on said objects such that the distance between epipolar lines associated with substantially identical features is less than the distance between corresponding locations of two neighboring features; (d) capturing a 2D image of said objects having said projected coded light pattern projected thereupon, said 2D image comprising reflected said feature types; and (e) extracting: (i) said reflected feature types according to the unique bi-dimensional formations; and (ii) locations of said reflected feature types on respective said epipolar lines in said 2D image.

Claims

1. A method of obtaining data from a 2D (two-dimensional) image in order to determine the 3D (three-dimensional) shape of objects appearing in said 2D image, said method comprising:

providing a first bi-dimensional coded light pattern having a plurality of feature types, each feature type being distinguishable according to a unique bi-dimensional formation and each feature type comprising a plurality of elements having varying light intensity, wherein said plurality of elements comprises;

at least one maximum element;

at least one minimum element; and

at least one saddle element;

providing a second bi-dimensional coded light pattern which is the same as said first bi-dimensional coded light pattern with the one or more maximum and/or minimum elements inverted to minimum or maximum and in which the at least one saddle element remains unchanged, respectively;

projecting said first bi-dimensional coded light pattern and said second bi-dimensional coded light pattern on said objects;

capturing;

i) a first 2D image of said objects, having said first projected bi-dimensional coded light pattern projected thereupon, said first 2D image comprising reflected said feature types; and

ii) a second 2D image of said objects, having said second bi-dimensional coded light pattern projected thereupon, said second 2D image comprising reflected said feature types;

obtaining a resultant image from the subtraction of said second 2D image from said first 2D image, the resultant image comprising maximum, minimum and saddle points resulting from the subtraction; and

extracting from said resultant image, feature types'"'"' elements locations in said first and/or second image, based on the maximum, saddle and/or minimum elements locations in said resultant image.

2. The method according to claim 1, further comprising determining feature types in the first and/or second images based on feature types'"'"' elements locations.

3. The method according to claim 1, further comprising determining corresponding 3D spatial coordinates of the objects by determining locations of the one or more of the following:

at least one maximum element, at least one minimum element, and at least one saddle element and/or by determining locations of the non-maximum and non-minimum elements on the respective epipolar lines.

4. The method according to claim 1, further comprising determining corresponding 3D spatial coordinates of the objects based on the feature types'"'"' elements locations in said resultant image.

5. The method according to claim 1, further comprising projecting the light patterns by one or more of the following:

a) temporally;

b) with differing spectral values of light; and

c) with differing polarization.

6. The method according to claim 1, further comprising capturing the first 2D image and second 2D images temporally, giving rise to temporal imaging.

7. The method according to claim 6, further comprising carrying out the temporal imaging over non-uniform time intervals.

8. The method according to claim 1, further comprising capturing the first and second 2D images simultaneously by using spectral differentiation.

9. The method according to claim 1, further comprising carrying out the extraction of the feature types'"'"' elements locations in the resultant image by determining intensity values.

10. An apparatus configured to obtain data from a 2D (two-dimensional) image in order to determine the 3D (three-dimensional) shape of objects appearing in said 2D image, said apparatus comprising:

a first bi-dimensional coded light pattern having a plurality of feature types, each feature type being distinguishable according to a unique bi-dimensional formation and each feature type comprising a plurality of elements having varying light intensity, wherein said plurality of elements comprises;

a) at least one maximum element;

b) at least one minimum element; and

c) at least one saddle element;

a second bi dimensional light pattern which is the same as said first bi-dimensional coded pattern with the one or more maximum and/or minimum elements inverted to minimum or maximum and in which the at least one saddle element remains unchanged, respectively;

a projection module configured to project said first bi-dimensional coded light pattern and said second bi-dimensional light pattern on said objects;

at least one imaging module configured to capture a first 2D image of said objects, having said first projected coded light pattern projected thereupon, said first 2D image comprising reflected said feature types, and said at least one imaging module is configured to capture a second 2D image of said objects, having said second coded light pattern projected thereupon, said second 2D image comprising reflected said feature types; and

an image processing module configured to;

i) obtain a resultant image from the subtraction of said second 2D image from said first 2D image, said resultant image comprising maximum, minimum and saddle points resulting from the subtraction; and

ii) extract from said resultant image, feature types'"'"' elements locations in said first and/or second image, based on the maximum, saddle and/or minimum elements locations in said resultant image.

11. The apparatus according to claim 10, wherein the image processing module is adapted to determine feature types in said first and/or second images based on feature types'"'"' elements locations.

12. The apparatus according to claim 10, wherein the projection module is adapted to project the light patterns by one or more of the following:

a) temporally;

b) with differing spectral values of light; and

c) with differing polarization.

13. The apparatus according to claim 10, wherein the at least one imaging module is configured to capture the first 2D image and the second 2D image temporally, giving rise to temporal imaging.

14. The apparatus according to claim 13, wherein the at least one imaging module is configured to carry out the temporal imaging over non-uniform time intervals.

15. The apparatus according to claim 10, wherein the at least one imaging module is configured to capture the first and the second 2D images, simultaneously by using spectral differentiation.

16. The apparatus according to claim 10, wherein the image processing module is configured to carry out the extraction of the feature types in the resultant image by determining intensity values of sample points.