Depth ranging with Moiré patterns

• US 8,462,207 B2
• Filed: 02/11/2010
• Issued: 06/11/2013
• Est. Priority Date: 02/12/2009
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: PrimeSense Ltd (Apple Inc.)
• Inventors: Garcia, Javier, Shpunt, Alexander
• Primary Examiner(s): Perungavoor, Sath V
• Assistant Examiner(s): WILLIAMS, JEFFERY A
• Application Number:
  US12/703,794
• Publication Number:
  US 20100201811A1
• Time in Patent Office: 1,216 Days
• Field of Search: 348/135, 348/136, 348/137
• US Class Current: 348/136
• CPC Class Codes: G01B 11/2513 with several lines being projected in more than one direction, e.g. grids, patterns
• CPC Class Codes: G01B 11/254 Projection of a pattern, viewing through a pattern, e.g. moiré

Abstract

A method for three-dimensional mapping of an object, including projecting with a projector a set of fringes on the object and capturing an image of the object in a camera. The method further includes processing the captured image so as to detect a Moiré pattern associated with the object and so as to extract depth information from the Moiré pattern, and configuring the projector and the camera so that a locally unambiguous characteristic of the Moiré pattern is related to a depth of the object.

Claims

1. A method for three-dimensional mapping of an object, comprising:

projecting with a projector a set of fringes on the object;

capturing an image of the object in a camera, wherein the image contains a Moiré

pattern associated with the object, and a visibility of the Moiré

pattern varies with a distance of the object from the camera,

wherein the visibility of the pattern in a region is defined as a relation between a maximum intensity in the region and a minimum intensity in the region; and

processing the captured image so as to detect the Moiré

pattern associated with the object and so as to extract depth information from the visibility of the Moiré

pattern.

2. The method according to claim 1, wherein the set of fringes comprises a single set of fringes diverging from an aperture of the projector.

3. The method according to claim 2, wherein processing the captured image comprises multiplying the captured image with a reference grating to form a composite image comprising the Moiré

pattern.

4. The method according to claim 2, wherein processing the captured image comprises multiplying the captured image with a digital representation of a reference grating to form a composite image comprising the Moiré

pattern.

5. The method according to claim 1, wherein processing the captured image comprises identifying and removing a reflectivity component of the object in the captured image.

6. The method according to claim 5, wherein the set of fringes have a spatial frequency, and wherein identifying and removing the reflectivity component comprises filtering a composite image derived from the captured image around a frequency corresponding to twice the spatial frequency.

7. The method according to claim 1, wherein the set of fringes have a spatial frequency, and wherein detecting the Moiré

pattern comprises filtering a composite image derived from the captured image in a low pass filter blocking frequencies greater than and including a frequency corresponding to the spatial frequency.

8. The method according to claim 1, wherein the set of fringes have a spatial period at the object, and wherein an effective displacement of the fringes is less than the spatial period.

9. The method according to claim 1, wherein projecting the set of fringes comprises projecting the set of fringes via a beamsplitter, and wherein capturing the image comprises capturing radiation from the object via the beamsplitter, and wherein detecting the Moiré

pattern comprises orienting the beamsplitter to form the Moiré

pattern.

10. The method according to claim 1, wherein the set of fringes comprises a first set of fringes and a second set of fringes configured to generate the Moiré

pattern.

11. The method according to claim 10, wherein the set of fringes have a spatial frequency, and wherein processing the captured image comprises identifying and removing a reflectivity component by filtering a composite image derived from the captured image at a frequency corresponding to the spatial frequency.

12. The method according to claim 1, wherein processing the image comprises extracting the depth information from a function of an intensity of the Moiré

pattern.

13. The method according to claim 12, wherein the function comprises the intensity.

14. The method according to claim 1, wherein the visibility of the Moiré

pattern varies monotonically with locations on the object.

15. The method according to claim 1, wherein the visibility of the Moiré

pattern varies non-monotonically with locations on the object.

16. The method according to claim 1, wherein projecting the set of fringes comprises configuring the projector to use Young'"'"'s method to generate the set of fringes.

17. The method according to claim 1, wherein the projector comprises a single projection lens having two numerical apertures with a separation therebetween selected to provide the fringes with a given spatial frequency and a given visibility variation with distance.

18. The method according to claim 1, wherein the projector comprises a cylindrical lens array.

19. Apparatus for three-dimensional mapping of an object, comprising:

a projector which is configured to project a single set of fringes on the object;

a camera which is configured to capture an image of the object, wherein the image contains a Moiré

pattern associated with the object, and a visibility of the Moiré

pattern varies with a distance of the object from the camera,

wherein the visibility of the pattern in a region is defined as a relation between a maximum intensity in the region and a minimum intensity in the region; and

a processor which is configured to process the captured image so as to detect the Moiré

pattern associated with the object and so as to extract depth information from the visibility of the Moiré

pattern.

20. The apparatus according to claim 19, and comprising a reference grating located at an image plane of the camera, and wherein processing the captured image comprises multiplying the captured image with the reference grating to form a composite image comprising the Moiré

pattern.

21. The apparatus according to claim 19, wherein processing the captured image comprises multiplying the captured image with a digital representation of a reference grating to form a composite image comprising the Moiré

pattern.

22. The apparatus according to claim 19, wherein the single set of fringes have a frequency of repetition, and comprising identifying and removing a reflectivity component of the object in the captured image by filtering a composite image derived from the captured image at a frequency corresponding to twice the frequency of repetition.

23. The apparatus according to claim 19, wherein the single set of fringes have a spatial frequency, and wherein detecting the Moiré

pattern comprises filtering a composite image derived from the captured image in a low pass filter blocking frequencies greater than and including a frequency corresponding to the spatial frequency.

24. The apparatus according to claim 19, wherein the single set of fringes have a spatial period at the object, and wherein an effective displacement of the fringes is less than the spatial period.

25. The apparatus according to claim 19, and comprising a beamsplitter, wherein projecting the single set of fringes comprises projecting the fringes via the beamsplitter, and wherein capturing the image comprises capturing radiation from the object via the beamsplitter, and wherein detecting the Moiré

pattern comprises orienting the beamsplitter to form the Moiré

pattern.

26. The apparatus according to claim 19, wherein the processor is configured to extract the depth information from a function of an intensity of the Moiré

pattern.

27. The apparatus according to claim 19, wherein the visibility of the Moiré

pattern varies monotonically with locations on the object.

28. The apparatus according to claim 19, wherein the visibility of the Moiré

pattern varies non-monotonically with locations on the object.

29. Apparatus for three-dimensional mapping of an object, comprising:

a projector which is configured to project a first set of fringes and a second set of fringes on the object;

a camera which is configured to capture an image of the object, wherein the image contains a Moiré

pattern associated with the object, and a visibility of the Moiré

pattern varies with a distance of the object from the camera,

wherein the visibility of the pattern in a region is defined as a relation between a maximum intensity in the region and a minimum intensity in the region; and

a processor which is configured to process the captured image so as to detect the Moiré

pattern associated with the object and so as to extract depth information from the visibility of the Moiré

pattern.

30. The apparatus according to claim 29, wherein the first set of fringes and the second set of fringes are configured to generate the Moiré

pattern.

31. The apparatus according to claim 29, wherein the first and second sets of fringes have a spatial frequency, and wherein processing the captured image comprises identifying and removing a reflectivity component by filtering a composite image derived from the captured image at a frequency corresponding to the spatial frequency.

32. The apparatus according to claim 29, wherein the the processor is configured to extract the depth information from comprises a function of an intensity of the Moiré

pattern.

33. The apparatus according to claim 29, wherein the visibility of the Moiré

pattern varies monotonically with locations on the object.

34. The apparatus according to claim 29, wherein the visibility of the Moiré

pattern varies non-monotonically with locations on the object.