Depth-varying light fields for three dimensional sensing

• US 8,374,397 B2
• Filed: 03/09/2011
• Issued: 02/12/2013
• Est. Priority Date: 10/11/2005
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: PrimeSense Ltd (Apple Inc.)
• Inventors: Shpunt, Alexander, Zalevsky, Zeev
• Primary Examiner(s): Carter, Aaron W
• Application Number:
  US13/043,488
• Publication Number:
  US 20110158508A1
• Time in Patent Office: 706 Days
• Field of Search: 382/154, 382/285, 345419-427, 356 12- 14, 348 42- 60, 359462-477, 352 57- 65, 33/204, 353 7- 9, 378 41- 42, 396324-331
• US Class Current: 382/106
• CPC Class Codes: G01B 11/25 by projecting a pattern, e.g. one or more lines, moiré fringes on the object G01B11/255 takes precedence ; image analysis for depth or shape recovery G06T7/50
• CPC Class Codes: G01B 11/2513 with several lines being projected in more than one direction, e.g. grids, patterns
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects

Abstract

A method for mapping includes projecting onto an object a pattern of multiple spots having respective positions and shapes, such that the positions of the spots in the pattern are uncorrelated, while the shapes share a common characteristic. An image of the spots on the object is captured and processed so as to derive a three-dimensional (3D) map of the object.

Claims

1. A method for mapping, comprising:

projecting onto an object a pattern of multiple spots having respective positions and shapes, such that the positions of the spots in the pattern are uncorrelated, while the shapes share a common characteristic, which varies as a function of distance from a source of the pattern;

capturing an image of the spots on the object; and

processing the image so as to derive a three-dimensional (3D) map of the object by analyzing both the respective positions of the spots and the characteristic of the shapes of the spots on a surface of the object in the image.

2. The method according to claim 1, wherein the pattern of spots comprises a random speckle pattern.

3. The method according to claim 1, wherein projecting the pattern comprises passing a beam of coherent light through at least one optical element, which is designed to superpose a first optical constraint that splits the beam into the multiple spots, and a second optical constraint for applying the common characteristic of the shapes to at least one spot.

4. The method according to claim 3, wherein the at least one optical element comprises a first optical element for creating the pattern and a second optical element for applying the common characteristic.

5. The method according to claim 1, wherein the spots share an elongate shape, which rotates as a function of distance from a source of the pattern, and wherein analyzing the characteristic comprises determining a direction of the spots on the surface of the object.

6. The method according to claim 1, wherein processing the image comprises analyzing the characteristic of the spots on a surface of the object in the image so as to determine the distance of the surface from the source, and finding respective offsets between the pattern on multiple areas of the object in the image and the pattern in a reference image, and using the offsets together with the distance to derive the 3D map.

7. The method according to claim 6, wherein finding the respective offsets comprises choosing the reference image from among a plurality of reference images responsively to the distance of the surface from the source.

8. The method according to claim 1, wherein the spots in the pattern have an elongate shape, which is aligned in a first direction, and wherein processing the image comprises finding respective offsets in a second direction, perpendicular to the first direction, between the pattern on multiple areas of the object in the image and the pattern in a reference image so as to derive the 3D map.

9. The method according to claim 8, wherein projecting the pattern of spots comprises passing a beam of coherent light through a diffuser, wherein the beam has a profile at the diffuser that is elongated in the second direction.

10. The method according to claim 1, wherein capturing the image comprises capturing a succession of images while the object is moving, and wherein processing the image comprises tracking a movement of the object by processing the succession of the images.

11. The method according to claim 10, wherein the object is a part of a human body, and wherein tracking the movement comprises identifying a gesture made by the part of the human body and providing an input to a computer application responsively to the gesture.

12. Apparatus for mapping, comprising:

an illumination assembly, which is configured to project onto an object a pattern of multiple spots having respective positions and shapes, such that the positions of the spots in the pattern are uncorrelated, while the shapes share a common characteristic, which varies as a function of distance from a source of the pattern;

an imaging assembly, which is configured to capture an image of the spots on the object; and

an image processor, which is coupled to process the image so as to derive a three-dimensional (3D) map of the object by analyzing both the respective positions of the spots and the characteristic of the shapes of the spots on a surface of the object in the image.

13. The apparatus according to claim 12, wherein the pattern of spots comprises a random speckle pattern.

14. The apparatus according to claim 12, wherein the illumination assembly comprises:

at least one optical element, which is designed to superpose a first optical constraint that splits the beam into the multiple spots, and a second optical constraint for applying the common characteristic of the shapes to at least one spot; and

a light source, which is configured to direct a beam of coherent light through the at least one optical element.

15. The apparatus according to claim 14, wherein the at least one optical element comprises a first optical element for creating the pattern and a second optical element for applying the common characteristic to the shapes.

16. The apparatus according to claim 12, wherein the spots share an elongate shape, which rotates as a function of distance from a source of the pattern, and wherein the image processor is configured to determine a direction of the spots on the surface of the object.

17. The apparatus according to claim 12, wherein the image processor is configured to analyze the characteristic of the spots on a surface of the object in the image so as to determine the distance of the surface from the source, and to find respective offsets between the pattern on multiple areas of the object in the image and the pattern in a reference image, and to use the offsets together with the distance to derive the 3D map.

18. The apparatus according to claim 17, wherein the image processor is configured to choose the reference image from among a plurality of reference images responsively to the distance of the surface from the source.

19. The apparatus according to claim 12, wherein the spots in the pattern have an elongate shape, which is aligned in a first direction, and wherein the image processor is configured to find respective offsets in a second direction, perpendicular to the first direction, between the pattern on multiple areas of the object in the image and the pattern in a reference image so as to derive the 3D map.

20. The apparatus according to claim 19, wherein the illumination assembly comprises a diffuser and a light source, which is configured to direct a beam of coherent light through the diffuser, wherein the beam has a profile at the diffuser that is elongated in the second direction.

21. The apparatus according to claim 12, wherein the imaging assembly is configured to capture a succession of images while the object is moving, and wherein the image processor is configured to track a movement of the object by processing the succession of the images.

22. The apparatus according to claim 21, wherein the object is a part of a human body, and wherein the image processor is configured to identify a gesture made by the part of the human body and to provide an input to a computer application responsively to the gesture.