Depth mapping using projected patterns

• US 20080240502A1
• Filed: 09/06/2007
• Published: 10/02/2008
• Est. Priority Date: 04/02/2007
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: PrimeSense Ltd (Apple Inc.)
• Inventors: Shpunt, Alexander, Arieli, Yoel, Machline, Meir, Freedman, Barak
• Granted Patent: US 8,150,142 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G01B 11/2513 with several lines being projected in more than one direction, e.g. grids, patterns
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06V 10/145 Illumination specially adapted for pattern recognition, e.g. using gratings
• CPC Class Codes: G06V 2201/12 Acquisition of 3D measurements of objects

Abstract

Apparatus for mapping an object includes an illumination assembly, which includes a single transparency containing a fixed pattern of spots. A light source transilluminates the single transparency with optical radiation so as to project the pattern onto the object. An image capture assembly captures an image of the pattern that is projected onto the object using the single transparency. A processor processes the image captured by the image capture assembly so as to reconstruct a three-dimensional (3D) map of the object.

Claims

1. Apparatus for mapping an object, comprising:

an illumination assembly, comprising;

a single transparency containing a fixed pattern of spots; and

a light source, which is configured to transilluminate the single transparency with optical radiation so as to project the pattern onto the object;

an image capture assembly, which is configured to capture an image of the pattern that is projected onto the object using the single transparency; and

a processor, which is coupled to process the image captured by the image capture assembly so as to reconstruct a three-dimensional (3D) map of the object.

2. The apparatus according to claim 1, wherein the pattern is uncorrelated over a range of depths that is mapped by the apparatus.

3. The apparatus according to claim 1, wherein the processor is arranged to derive the 3D map by finding respective offsets between the pattern of the spots on multiple areas of the object captured in the image of the pattern that is projected onto the object and a reference image of the pattern, wherein the respective offsets are indicative of respective distances between the areas and the image capture assembly.

4. The apparatus according to claim 1, wherein the image capture assembly is arranged to capture images of the pattern on the object from a single, fixed location and angle relative to the illumination assembly.

5. The apparatus according to claim 4, wherein the transparency and light source are fixed in respective positions in the illumination assembly.

6. The apparatus according to claim 5, wherein the processor is configured to reconstruct the 3D map using the images that are captured only from the single, fixed location and angle with the transparency and light source only in the respective positions.

7. The apparatus according to claim 1, wherein the light source comprises a point source of the optical radiation.

8. The apparatus according to claim 1, wherein the light source comprises a light-emitting diode (LED).

9. The apparatus according to claim 1, wherein the processor is arranged to process a succession of images captured while the object is moving so as to map a 3D movement of the object.

10. The apparatus according to claim 9, wherein the object is a part of a human body, and wherein the 3D movement comprises a gesture made by the part of the human body, and wherein the processor is coupled to provide an input to a computer application responsively to the gesture.

11. Apparatus for mapping an object, comprising:

an illumination assembly, comprising;

a transparency containing an uncorrelated pattern of spots; and

a light source, which is configured to transilluminate the transparency with optical radiation so as to project the uncorrelated pattern onto the object;

an image capture assembly, which is configured to capture an image of the uncorrelated pattern that is projected onto the object; and

a processor, which is coupled to process the image captured by the image capture assembly so as to reconstruct a three-dimensional (3D) map of the object.

12. The apparatus according to claim 11, wherein the uncorrelated pattern comprises a pseudo-random pattern.

13. The apparatus according to claim 11, wherein the uncorrelated pattern comprises a quasi-periodic pattern.

14. The apparatus according to claim 13, wherein the quasi-periodic pattern has an n-fold symmetry, with n=5 or n≧

7.

15. The apparatus according to claim 11, wherein the uncorrelated pattern has a duty cycle that is less than 1/e.

16. The apparatus according to claim 11, wherein the spots have a local duty cycle that varies across the pattern.

17. The apparatus according to claim 11, wherein the transparency contains a plurality of parallel bands, repeating periodically in a first direction, each band containing a replica of the uncorrelated pattern extending across at least a part of the transparency in a second direction, perpendicular to the first direction.

18. The apparatus according to claim 11, wherein the processor is configured to derive the 3D map by finding respective offsets between the pattern of the spots on multiple areas of the object captured in the image of the pattern that is projected onto the object and a reference image of the pattern, wherein the respective offsets are indicative of respective distances between the areas and the image capture assembly.

19. The apparatus according to claim 18, wherein the spots have a local duty cycle that varies monotonically along an axis across the pattern, and wherein the processor is configured to determine local gray levels of the multiple areas in the image responsively to the local duty cycle, and to estimate the respective offsets based on the local gray levels.

20. A method for mapping an object, comprising:

transilluminating a single transparency containing a fixed pattern of spots so as to project the pattern onto the object;

capturing an image of the pattern that is projected onto the object using the single transparency; and

processing the captured image so as to reconstruct a three-dimensional (3D) map of the object.

21. The method according to claim 20, wherein the pattern is uncorrelated over a range of depths that is mapped by the method.

22. The method according to claim 20, wherein processing the captured image comprises finding respective offsets between the pattern of the spots on multiple areas of the object captured in the image of the pattern that is projected onto the object and a reference image of the pattern, and determining respective distances to the areas responsively to the respective offsets.

23. The method according to claim 20, wherein capturing the image comprises capturing images of the pattern on the object from a single, fixed location and angle relative to the single transparency.

24. The method according to claim 23, wherein transilluminating the transparency comprises projecting optical radiation through the transparency from a light source, wherein the transparency and light source are fixed in respective positions relative to one another.

25. The method according to claim 24, wherein processing the captured image comprises reconstructing the 3D map without use of further images captured from any location or angle other than the single, fixed location and angle and without movement of the transparency and the light source from the fixed respective positions.

26. The method according to claim 20, wherein transilluminating the single transparency comprises projecting optical radiation from a point source through the single transparency.

27. The method according to claim 20, wherein transilluminating the single transparency comprises projecting optical radiation from a light-emitting diode (LED) through the single transparency.

28. The method according to claim 20, wherein processing the captured image comprises processing a succession of images captured while the object is moving so as to map a 3D movement of the object.

29. The method according to claim 28, wherein the object is a part of a human body, and wherein the 3D movement comprises a gesture made by the part of the human body, and wherein processing the succession of the images comprises providing an input to a computer application responsively to the gesture.

30. A method for mapping an object, comprising:

an illumination assembly, comprising;

transilluminating a transparency containing an uncorrelated pattern of spots so as to project the uncorrelated pattern onto the object;

capturing an image of the uncorrelated pattern that is projected onto the object; and

processing the captured image so as to reconstruct a three-dimensional (3D) map of the object.

31. The method according to claim 30, wherein the uncorrelated pattern comprises a pseudo-random pattern.

32. The method according to claim 30, wherein the uncorrelated pattern comprises a quasi-periodic pattern.

33. The method according to claim 32, wherein the quasi-periodic pattern has an n-fold symmetry, with n=5 or n≧

7.

34. The method according to claim 30, wherein the uncorrelated pattern has a duty cycle that is less than 1/e.

35. The method according to claim 30, wherein the spots have a local duty cycle that varies across the pattern.

36. The method according to claim 30, wherein the transparency contains a plurality of parallel bands, repeating periodically in a first direction, each band containing a replica of the uncorrelated pattern extending across at least a part of the transparency in a second direction, perpendicular to the first direction.

37. The method according to claim 30, wherein processing the captured image comprises finding respective offsets between the pattern of the spots on multiple areas of the object captured in the image of the pattern that is projected onto the object and a reference image of the pattern, and determining respective distances to the areas responsively to the respective offsets.

38. The method according to claim 37, wherein the spots have a local duty cycle that varies monotonically along an axis across the pattern, and wherein finding the respective offsets comprises determining local gray levels of the multiple areas in the image responsively to the local duty cycle, and estimating the respective offsets based on the local gray levels.