Depth-varying light fields for three dimensional sensing

• US 20080106746A1
• Filed: 03/13/2007
• Published: 05/08/2008
• Est. Priority Date: 10/11/2005
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: PrimeSense Ltd (Apple Inc.)
• Inventors: Shpunt, Alexander, Zalevsky, Zeev
• Granted Patent: US 8,050,461 B2
• Time in Patent Office: Days
• US Class Current: 356/610
• CPC Class Codes: G01B 11/2513 with several lines being projected in more than one direction, e.g. grids, patterns
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps

Abstract

A method for mapping includes projecting onto an object a pattern of multiple spots having respective positions and shapes, such that the positions of the spots in the pattern are uncorrelated, while the shapes share a common characteristic. An image of the spots on the object is captured and processed so as to derive a three-dimensional (3D) map of the object.

Claims

1. A method for mapping, comprising:

projecting onto an object a pattern of multiple spots having respective positions and shapes, such that the positions of the spots in the pattern are uncorrelated, while the shapes share a common characteristic;

capturing an image of the spots on the object; and

processing the image so as to derive a three-dimensional (3D) map of the object.

2. The method according to claim 1, wherein the pattern of spots comprises a random speckle pattern.

3. The method according to claim 1, wherein projecting the pattern comprises passing a beam of coherent light through at least one optical element, which is designed to superpose a first optical constraint that splits the beam into the multiple spots, and a second optical constraint for applying the common characteristic of the shapes to at least one spot.

4. The method according to claim 3, wherein the at least one optical element comprises a first optical element for creating the pattern and a second optical element for applying the common characteristic.

5. The method according to claim 1, wherein the common characteristic of the shapes varies as a function of distance from a source of the pattern, and wherein processing the image comprises analyzing the characteristic of the spots on a surface of the object in the image so as to determine the distance of the surface from the source.

6. The method according to claim 5, wherein the spots share an elongate shape, which rotates as a function of distance from a source of the pattern, and wherein analyzing the characteristic comprises determining a direction of the spots on the surface of the object.

7. The method according to claim 5, wherein processing the image comprises finding respective offsets between the pattern on multiple areas of the object in the image and the pattern in a reference image, and using the offsets together with the distance to derive the 3D map.

8. The method according to claim 7, wherein finding the respective offsets comprises choosing the reference image from among a plurality of reference images responsively to the distance of the surface from the source.

9. The method according to claim 1, wherein the spots in the pattern have an elongate shape, which is aligned in a first direction, and wherein processing the image comprises finding respective offsets in a second direction, perpendicular to the first direction, between the pattern on multiple areas of the object in the image and the pattern in a reference image so as to derive the 3D map.

10. The method according to claim 9, wherein projecting the pattern of spots comprises passing a beam of coherent light through a diffuser, wherein the beam has a profile at the diffuser that is elongated in the second direction.

11. The method according to claim 1, wherein capturing the image comprises capturing a succession of images while the object is moving, and wherein processing the image comprises tracking a movement of the object by processing the succession of the images.

12. The method according to claim 11, wherein the object is a part of a human body, and wherein tracking the movement comprises identifying a gesture made by the part of the human body and providing an input to a computer application responsively to the gesture.

13. A method for imaging, comprising:

defining a first optical constraint such that application of the first optical constraint to a beam of light splits the beam into a pattern of multiple spots;

defining a second optical constraint such that application of the second optical constraint to the beam of light causes the beam to form a spot having a shape characteristic that changes in a predefined manner as a function of a distance along an axis of the beam;

designing at least one optical element so as to superpose the first and second optical constraints; and

directing a beam of light through the at least one optical element so as to project the pattern onto a surface such that the multiple spots in the pattern have the shape characteristic.

14. The method according to claim 13, wherein the at least one optical element comprises a first optical element for splitting the beam into the pattern and a second optical element for applying the shape characteristic.

15. The method according to claim 14, wherein the pattern comprises a speckle pattern.

16. The method according to claim 15, wherein the first optical element comprises a diffuser.

17. The method according to claim 13, wherein the at least one optical element comprises a diffractive optical element (DOE).

18. The method according to claim 17, wherein the DOE comprises at least one zone plate for imparting an elongate shape to the spots.

19. The method according to claim 18, wherein the at least one zone plate comprises a plurality of superposed zone plates, having different, respective periods and angular orientations so as to cause the elongate shape of the spots to rotate as a function of the distance.

20. The method according to claim 13, wherein the at least one optical element comprises a refractive optical element.

21. The method according to claim 13, wherein the pattern defined by the first optical constraint has a duty cycle that is no greater than ¼

.

22. The method according to claim 13, wherein the pattern defines respective positions of the spots such that the positions are uncorrelated.

23. The method according to claim 13, wherein the second optical constraint causes the spot to have an elongate shape, which rotates as a function of the distance.

24. The method according to claim 13, wherein the second optical constraint causes the spot to have an annular shape.

25. The method according to claim 13, and comprising capturing an image of the spots on the surface, and processing the image so as to determine the distance of the surface from the at least one optical element.

26. Apparatus for mapping, comprising:

an illumination assembly, which is configured to project onto an object a pattern of multiple spots having respective positions and shapes, such that the positions of the spots in the pattern are uncorrelated, while the shapes share a common characteristic;

an imaging assembly, which is configured to capture an image of the spots on the object; and

an image processor, which is coupled to process the image so as to derive a three-dimensional (3D) map of the object.

27. The apparatus according to claim 26, wherein the pattern of spots comprises a random speckle pattern.

28. The apparatus according to claim 26, wherein the illumination assembly comprises:

at least one optical element, which is designed to superpose a first optical constraint that splits the beam into the multiple spots, and a second optical constraint for applying the common characteristic of the shapes to at least one spot; and

a light source, which is configured to direct a beam of coherent light through the at least one optical element.

29. The apparatus according to claim 28, wherein the at least one optical element comprises a first optical element for creating the pattern and a second optical element for applying the common characteristic to the shapes.

30. The apparatus according to claim 26, wherein the common characteristic of the shapes varies as a function of distance from a source of the pattern, and wherein the image processor is configured to analyze the characteristic of the spots on a surface of the object in the image so as to determine the distance of the surface from the source.

31. The apparatus according to claim 30, wherein the spots share an elongate shape, which rotates as a function of distance from a source of the pattern, and wherein the image processor is configured to determine a direction of the spots on the surface of the object.

32. The apparatus according to claim 30, wherein the image processor is configured to find respective offsets between the pattern on multiple areas of the object in the image and the pattern in a reference image, and to use the offsets together with the distance to derive the 3D map.

33. The apparatus according to claim 32, wherein the image processor is configured to choose the reference image from among a plurality of reference images responsively to the distance of the surface from the source.

34. The apparatus according to claim 26, wherein the spots in the pattern have an elongate shape, which is aligned in a first direction, and wherein the image processor is configured to find respective offsets in a second direction, perpendicular to the first direction, between the pattern on multiple areas of the object in the image and the pattern in a reference image so as to derive the 3D map.

35. The apparatus according to claim 34, wherein the illumination assembly comprises a diffuser and a light source, which is configured to direct a beam of coherent light through the diffuser, wherein the beam has a profile at the diffuser that is elongated in the second direction.

36. The apparatus according to claim 26, wherein the imaging assembly is configured to capture a succession of images while the object is moving, and wherein the image processor is configured to track a movement of the object by processing the succession of the images.

37. The apparatus according to claim 36, wherein the object is a part of a human body, and wherein the image processor is configured to identify a gesture made by the part of the human body and to provide an input to a computer application responsively to the gesture.

38. Apparatus for imaging, comprising:

at least one optical element, which is designed so as to superpose first and second optical constraints, such that application of the first optical constraint to a beam of light splits the beam into a pattern of multiple spots, and application of the second optical constraint to the beam of light causes the beam to form a spot having a shape characteristic that changes in a predefined manner as a function of a distance along an axis of the beam; and

a light source, which is configured to direct a beam of light through the at least one optical element so as to project the pattern onto a surface such that the multiple spots in the pattern have the shape characteristic.

39. The apparatus according to claim 38, wherein the at least one optical element comprises a first optical element for splitting the beam into the pattern and a second optical element for applying the shape characteristic.

40. The apparatus according to claim 39, wherein the pattern comprises a speckle pattern.

41. The apparatus according to claim 40, wherein the first optical element comprises a diffuser.

42. The apparatus according to claim 38, wherein the at least one optical element comprises a diffractive optical element (DOE).

43. The apparatus according to claim 42, wherein the DOE comprises at least one zone plate for imparting an elongate shape to the spots.

44. The apparatus according to claim 43, wherein the at least one zone plate comprises a plurality of superposed zone plates, having different, respective periods and angular orientations so as to cause the elongate shape of the spots to rotate as a function of the distance.

45. The apparatus according to claim 38, wherein the at least one optical element comprises a refractive optical element.

46. The apparatus according to claim 38, wherein the pattern defined by the first optical constraint has a duty cycle that is no greater than ¼

.

47. The apparatus according to claim 38, wherein the pattern defines respective positions of the spots such that the positions are uncorrelated.

48. The apparatus according to claim 38, wherein the second optical constraint causes the spot to have an elongate shape, which rotates as a function of the distance.

49. The apparatus according to claim 38, wherein the second optical constraint causes the spot to have an annular shape.

50. The apparatus according to claim 38, and comprising an imaging assembly, which is configured to capture an image of the spots on the surface, and an image processor, which is coupled to process the image so as to determine the distance of the surface from the at least one optical element.