THREE-DIMENSIONAL SENSING USING SPECKLE PATTERNS

• US 20090096783A1
• Filed: 03/08/2007
• Published: 04/16/2009
• Est. Priority Date: 10/11/2005
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: PrimeSense Ltd (Apple Inc.)
• Inventors: Shpunt, Alexander, Zalevsky, Zeev
• Granted Patent: US 8,390,821 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G01B 11/25 by projecting a pattern, e.g. one or more lines, moiré fringes on the object G01B11/255 takes precedence ; image analysis for depth or shape recovery G06T7/50
• CPC Class Codes: G02B 5/18 Diffraction gratings holographic optical elements G02B5/32, G03H; integrally combined with optical fibres G02B6/02057; for coupling light guides G02B6/34; integrally combined with optical integrated light guides G02B6/12; grating systems G02B27/44
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06T 7/557 from light fields, e.g. from plenoptic cameras

Abstract

Apparatus (20) for 3D mapping of an object (28) includes an illumination assembly (30), including a coherent light source (32) and a diffuser (33), which are arranged to project a primary speckle pattern on the object. A single image capture assembly (38) is arranged to capture images of the primary speckle pattern on the object from a single, fixed location and angle relative to the illumination assembly. A processor (24) is coupled to process the images of the primary speckle pattern captured at the single, fixed angle so as to derive a 3D map of the object.

Claims

1. Apparatus for 3D mapping of an object, comprising:

an illumination assembly, comprising a coherent light source and a diffuser, which are arranged to project a primary speckle pattern on the object;

a single image capture assembly, which is arranged to capture images of the primary speckle pattern on the object from a single, fixed location and angle relative to the illumination assembly; and

a processor, which is coupled to process the images of the primary speckle pattern captured at the single, fixed angle so as to derive a 3D map of the object.

2. The apparatus according to claim 1, and comprising a mount, which is attached to the illumination assembly and the image capture assembly so as to hold the image capture assembly in a fixed spatial relation to the illumination assembly.

3. The apparatus according to claim 2, wherein the image capture assembly comprises:

an array of detector elements arranged in a rectilinear pattern defining first and second, mutually-perpendicular axes; and

objective optics, which have an entrance pupil and are arranged to focus the image onto the array,

wherein the illumination assembly and image capture assembly are aligned by the mount so as to define a device axis that is parallel to the first axis and passes through the entrance pupil and through a spot at which a beam emitted by the coherent light source passes through the diffuser.

4. The apparatus according to claim 3, wherein the processor is arranged to derive the 3D map by finding an offset along only the first axis between the primary speckle pattern captured in one or more of the images and a reference image of the primary speckle pattern.

5. The apparatus according to claim 1, wherein the processor is arranged to derive the 3D map by finding respective offsets between the primary speckle pattern on multiple areas of the object captured in one or more of the images and a reference image of the primary speckle pattern, wherein the respective offsets are indicative of respective distances between the areas and the image capture assembly.

6. The apparatus according to claim 5, wherein the image capture assembly is located at a predetermined spacing from the illumination assembly, and wherein the respective offsets are proportional to the respective distances in a ratio that is determined by the spacing.

7. The apparatus according to claim 6, wherein the primary speckle pattern projected by the illumination assembly comprises speckles having a characteristic size, and wherein the size of the speckles in the images varies across the image by a tolerance that depends on the spacing, and wherein the spacing is selected so as to maintain the tolerance within a predefined bound.

8. The apparatus according to claim 5, wherein the processor is arranged to relate the respective offsets to respective coordinates in the 3D map using a parametric model of distortion in the image capture assembly.

9. The apparatus according to claim 5, wherein the processor is arranged to find the respective offsets by finding an initial match between the primary speckle pattern in a first area of the object and a corresponding area of the reference image at a first offset relative to the first area, and to apply a region growing procedure, based on the first offset, to find the respective offsets of pixels adjacent to the first area.

10. The apparatus according to claim 1, wherein the processor is arranged to process a succession of the images captured while the object is moving so as to map a 3D movement of the object.

11. The apparatus according to claim 10, wherein the object is a part of a human body, and wherein the 3D movement comprises a gesture made by the part of the human body, and wherein the processor is coupled to provide an input to a computer application responsively to the gesture.

12. The apparatus according to claim 1, wherein the illumination assembly comprises a beam former, which is arranged to reduce a variation of a contrast of the speckle pattern created by the diffuser over a sensing volume of the apparatus.

13. The apparatus according to claim 12, wherein the beam former comprises a diffractive optical element (DOE).

14. The apparatus according to claim 13, wherein the beam former comprises a lens arranged to define a Fourier plane of the diffuser, and wherein the DOE is located in the Fourier plane.

15. The apparatus according to claim 12, wherein the beam former is arranged. to reduce a divergence of light emitted from the diffuser.

16. The apparatus according to claim 12, wherein the beam former is arranged to equalize an intensity of light emitted from the diffuser across a plane transverse to an optical axis of the illumination assembly.

17. The apparatus according to claim 1, wherein the processor comprises an optical correlator.

18. The apparatus according to claim 17, wherein the optical correlator comprises a diffractive optical element (DOE) containing a reference speckle pattern, and wherein the image capture assembly comprises a lenslet array, which is arranged to project multiple sub-images of the object onto the DOE so as to generate respective correlation peaks that are indicative of 3D coordinates of the object.

19. The apparatus according to claim 1, wherein the coherent light source has a coherence length that is less than 1 cm.

20. The apparatus according to claim 1, wherein the primary speckle pattern comprises speckles having a characteristic size, and wherein the illumination assembly is configured so as to permit the characteristic size of the speckles to be adjusted by varying a distance between the coherent light source and the diffuser.

21. A method for 3D mapping of an object, comprising:

illuminating an object with a beam of diffused coherent light from a light source so as to project a primary speckle pattern on the object;

capturing images of the primary speckle pattern on the object from a single, fixed location and angle relative to the light source; and

processing the images of the primary speckle pattern captured at the single, fixed angle so as to derive a 3D map of the object.

22. The method according to claim 21, wherein capturing the images comprises capturing the images using an image capture assembly, which is held in a fixed spatial relation to the light source while capturing the images.

23. The method according to claim 22, wherein the image capture assembly comprises an array of detector elements arranged in a rectilinear pattern defining first and second, mutually-perpendicular axes, and wherein the light source comprises a diffuser, and wherein capturing the images comprises aligning an entrance pupil of the image capture assembly with a spot at which a beam passes through the diffuser along a device axis that is parallel to the first axis.

24. The method according to claim 23, wherein processing the images comprises finding an offset along only the first axis between the primary speckle pattern captured in one or more of the images and a reference image of the primary speckle pattern.

25. The method according to claim 21, wherein processing the images comprises finding respective offsets between the primary speckle pattern on multiple areas of the object captured in one or more of the images and a reference image of the primary speckle pattern, wherein the respective offsets are indicative of respective distances between the areas and the image capture assembly.

26. The method according to claim 25, wherein the respective offsets are proportional to the respective distances in a ratio that is determined by a spacing of the fixed location from the light source.

27. The method according to claim 26, wherein the primary speckle pattern comprises speckles having a characteristic size, and wherein the size of the speckles in the images varies across the image by a tolerance that depends on the spacing, and wherein capturing the images comprises selecting the spacing so as to maintain the tolerance within a predefined bound.

28. The method according to claim 25, wherein finding the respective offsets comprises relating the respective offsets to respective coordinates in the 3D map using a parametric model of distortion in the image capture assembly.

29. The apparatus according to claim 25, wherein finding the respective offsets comprises finding an initial match between the primary speckle pattern in a first area of the object and a corresponding area of the reference image at a first offset relative to the first area, and applying a region growing procedure, based on the first offset, to find the respective offsets of pixels adjacent to the first area.

30. The method according to claim 21, wherein processing the images comprises processing a succession of the images captured while the object is moving so as to map a 3D movement of the object.

31. The method according to claim 30, wherein the object is a part of a human body, and wherein the 3D movement comprises a gesture made by the part of the human body, and processing the images comprises providing an input to a computer application responsively to the gesture.

32. The method according to claim 21, wherein illuminating the object comprises forming the beam so as to reduce a variation of a contrast of the speckle pattern created by the light source over a given sensing volume.

33. The method according to claim 32, wherein forming the beam comprises passing the beam through a diffractive optical element (DOE).

34. The method according to claim 33, wherein the light source comprises a diffuser, and wherein passing the beam comprises arranging the DOE in a Fourier plane of the diffuser.

35. The method according to claim 32, wherein forming the beam comprises reducing a divergence of the beam.

36. The method according to claim 32, wherein forming the beam comprises equalizing an intensity of the beam across a plane transverse to an optical axis of the light source.

37. The method according to claim 21, wherein processing the images comprises applying the images to an optical correlator.

38. The method according to claim 37, wherein the optical correlator comprises a diffractive optical element (DOE) containing a reference speckle pattern, and wherein capturing the images comprises projecting multiple sub-images of the object onto the DOE so as to generate respective correlation peaks that are indicative of 3D coordinates of the object.

39. The method according to claim 21, wherein the coherent light source has a coherence length that is less than 1 cm.

40. The method according to claim 21, wherein illuminating the object comprises passing the light from the coherent light source through a diffuser in order to create the primary speckle patter, and wherein the primary speckle pattern comprises speckles having a characteristic size, and wherein the method comprises adjusting the characteristic size of the speckles by varying a distance between the coherent light source and the diffuser.

41. Apparatus for 3D mapping of an object, comprising:

an illumination assembly, comprising a coherent light source, having a coherence length less than 1 cm, and a diffuser, which are arranged to project a primary speckle pattern on the object;

an image capture assembly, which is arranged to capture images of the primary speckle pattern on the object; and

a processor, which is coupled to process the images of the primary speckle pattern so as to derive a 3D map of the object.

42. The apparatus according to claim 41, wherein the coherence length of the coherent light source is less than 0.5 mm.

43. The apparatus according to claim 41, wherein the coherent light source has a divergence greater than 5°

.

44. The apparatus according to claim 41, wherein the primary speckle pattern comprises speckles having a characteristic size, and wherein the illumination assembly is configured so as to permit the characteristic size of the speckles to be adjusted by varying a distance between the coherent light source and the diffuser.