Depth mapping using projected patterns

• US 8,150,142 B2
• Filed: 09/06/2007
• Issued: 04/03/2012
• Est. Priority Date: 04/02/2007
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: PrimeSense Ltd (Apple Inc.)
• Inventors: Freedman, Barak, Shpunt, Alexander, Machline, Meir, Arieli, Yoel
• Primary Examiner(s): Carter, Aaron W
• Application Number:
  US11/899,542
• Publication Number:
  US 20080240502A1
• Time in Patent Office: 1,671 Days
• Field of Search: 382/154, 382/285, 345419-427, 356 12- 14, 348 42- 60, 359462-477, 352 57- 65, 33/204, 353 7- 9, 378 41- 42, 396324-331
• US Class Current: 382/154
• CPC Class Codes: G01B 11/2513 with several lines being projected in more than one direction, e.g. grids, patterns
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06V 10/145 Illumination specially adapted for pattern recognition, e.g. using gratings
• CPC Class Codes: G06V 2201/12 Acquisition of 3D measurements of objects

Abstract

Apparatus for mapping an object includes an illumination assembly, which includes a single transparency containing a fixed pattern of spots. A light source transilluminates the single transparency with optical radiation so as to project the pattern onto the object. An image capture assembly captures an image of the pattern that is projected onto the object using the single transparency. A processor processes the image captured by the image capture assembly so as to reconstruct a three-dimensional (3D) map of the object.

Claims

1. Apparatus for mapping an object, comprising:

an illumination assembly, comprising;

a transparency containing an uncorrelated pattern of spots having a spatial duty cycle that is less than 1/e; and

a light source, which is configured to transilluminate the transparency with optical radiation so as to project the uncorrelated pattern onto the object;

an image capture assembly, which is configured to capture, from a single, fixed location and angle relative to the illumination assembly, an image of the uncorrelated pattern that is projected onto the object using the single transparency; and

a processor, which is coupled to process the image captured by the image capture assembly only from the single, fixed location and angle relative to the illumination assembly so as to reconstruct a three-dimensional (3D) map of the object.

6. The apparatus according to claim 1, wherein the transparency and light source are fixed in respective positions in the illumination assembly.

7. The apparatus according to claim 6, wherein the processor is configured to reconstruct the 3D map using images that are captured only from the single, fixed location and angle with the transparency and light source only in the respective positions.

8. The apparatus according to claim 1, wherein the light source comprises a point source of the optical radiation.

9. The apparatus according to claim 1, wherein the light source comprises a light-emitting diode (LED).

10. The apparatus according to claim 1, wherein the processor is arranged to process a succession of images captured while the object is moving so as to map a 3D movement of the object.

11. The apparatus according to claim 10, wherein the object is a part of a human body, and wherein the 3D movement comprises a gesture made by the part of the human body, and wherein the processor is coupled to provide an input to a computer application responsively to the gesture.

12. The apparatus according to claim 1, wherein the uncorrelated pattern comprises a pseudo-random pattern.

13. The apparatus according to claim 1, wherein the uncorrelated pattern comprises a quasi-periodic pattern.

14. The apparatus according to claim 13, wherein the quasi-periodic pattern has an n-fold symmetry, with n=5 or n≧

7.

15. The apparatus according to claim 1, wherein the spatial duty cycle varies across the pattern.

16. The apparatus according to claim 1, wherein the processor is configured to derive the 3D map by finding respective offsets between the pattern of the spots on multiple areas of the object captured in the image of the pattern that is projected onto the object and a reference image of the pattern, wherein the respective offsets are indicative of respective distances between the areas and the image capture assembly.

17. The apparatus according to claim 16, wherein the spots have a local duty cycle that varies monotonically along an axis across the pattern, and wherein the processor is configured to determine local gray levels of the multiple areas in the image responsively to the local duty cycle, and to estimate the respective offsets based on the local gray levels.

2. Apparatus for mapping an object, comprising:

an illumination assembly, comprising;

a transparency containing an uncorrelated pattern of spots arranged in a plurality of parallel bands, repeating periodically in a first direction, each band containing a replica of the uncorrelated pattern extending across at least a part of the transparency in a second direction, perpendicular to the first direction; and

a light source, which is configured to transilluminate the transparency with optical radiation so as to project the uncorrelated pattern onto the object;

an image capture assembly, which is configured to capture, from a single, fixed location and angle relative to the illumination assembly, an image of the uncorrelated pattern that is projected onto the object using the single transparency; and

a processor, which is coupled to process the image captured by the image capture assembly only from the single, fixed location and angle relative to the illumination assembly so as to reconstruct a three-dimensional (3D) map of the object.

5. The apparatus according to claim 2, wherein the processor is arranged to derive the 3D map by finding respective offsets between the pattern of the spots on multiple areas of the object captured in the image of the pattern that is projected onto the object and a reference image of the pattern, wherein the respective offsets are indicative of respective distances between the areas and the image capture assembly.

3. A method for mapping an object, comprising:

transilluminating a transparency containing an uncorrelated pattern of spots having a spatial duty cycle that is less than 1/e so as to project the uncorrelated pattern onto the object;

capturing an image from a single, fixed location and angle relative to the transparency of the uncorrelated pattern that is projected onto the object; and

processing the image captured only from the single, fixed location so as to reconstruct a three-dimensional (3D) map of the object.

19. The method according to claim 3, wherein transilluminating the transparency comprises projecting optical radiation through the transparency from a light source, wherein the transparency and light source are fixed in respective positions relative to one another.

20. The method according to claim 19, wherein processing the captured image comprises reconstructing the 3D map without use of further images captured from any location or angle other than the single, fixed location and angle and without movement of the transparency and the light source from the fixed respective positions.

21. The method according to claim 3, wherein transilluminating the single transparency comprises projecting optical radiation from a point source through the single transparency.

22. The method according to claim 3, wherein transilluminating the single transparency comprises projecting optical radiation from a light-emitting diode (LED) through the single transparency.

23. The method according to claim 3, wherein processing the captured image comprises processing a succession of images captured while the object is moving so as to map a 3D movement of the object.

24. The method according to claim 23, wherein the object is a part of a human body, and wherein the 3D movement comprises a gesture made by the part of the human body, and wherein processing the succession of the images comprises providing an input to a computer application responsively to the gesture.

25. The method according to claim 3, wherein the uncorrelated pattern comprises a pseudo-random pattern.

26. The method according to claim 3, wherein the uncorrelated pattern comprises a quasi-periodic pattern.

27. The method according to claim 26, wherein the quasi-periodic pattern has an n-fold symmetry, with n=5 or n≧

7.

28. The method according to claim 3, wherein the spatial duty cycle varies across the pattern.

29. The method according to claim 3, wherein processing the captured image comprises finding respective offsets between the pattern of the spots on multiple areas of the object captured in the image of the pattern that is projected onto the object and a reference image of the pattern, and determining respective distances to the areas responsively to the respective offsets.

30. The method according to claim 29, wherein the spots have a local duty cycle that varies monotonically along an axis across the pattern, and wherein finding the respective offsets comprises determining local gray levels of the multiple areas in the image responsively to the local duty cycle, and estimating the respective offsets based on the local gray levels.

4. A method for mapping an object, comprising:

transilluminating a transparency containing an uncorrelated pattern of spots arranged in a plurality of parallel bands, repeating periodically in a first direction, each band containing a replica of the uncorrelated pattern extending across at least a part of the transparency in a second direction, perpendicular to the first direction, so as to project the uncorrelated pattern onto the object;

capturing an image from a single, fixed location and angle relative to the transparency of the uncorrelated pattern that is projected onto the object; and

processing the image captured only from the single, fixed location so as to reconstruct a three-dimensional (3D) map of the object.

18. The method according to claim 4, wherein processing the captured image comprises finding respective offsets between the pattern of the spots on multiple areas of the object captured in the image of the pattern that is projected onto the object and a reference image of the pattern, and determining respective distances to the areas responsively to the respective offsets.