Scanning projectors and image capture modules for 3D mapping

• US 9,098,931 B2
• Filed: 08/10/2011
• Issued: 08/04/2015
• Est. Priority Date: 08/11/2010
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Shpunt, Alexander, Pesach, Benny, Akerman, Ronen
• Primary Examiner(s): McDowell, Jr., Maurice L
• Application Number:
  US13/810,451
• Publication Number:
  US 20130127854A1
• Time in Patent Office: 1,455 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G01B 11/2518 Projection by scanning of the object
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 13/363 using image projection screens volumetric display H04N13/388
• CPC Class Codes: H04N 13/365 using digital micromirror devices [DMD]
• CPC Class Codes: H04N 13/398 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths

Abstract

Apparatus (20) for mapping includes an illumination module (30), which includes a radiation source (32), which is configured to emit a beam of radiation. A scanner (34) receives and scans the beam over a selected angular range. Illumination optics (35) project the scanned beam so as to create a pattern of spots extending over a region of interest. An imaging module (38) captures an image of the pattern that is projected onto an object (28) in the region of interest. A processor (46) processes the age in order to construct a three-dimensional (3D) map of the object.

Claims

1. Apparatus for mapping, comprising:

an illumination module, comprising;

a radiation source, which is configured to emit a beam of radiation;

a scanner, which is configured to receive and scan the beam over a selected angular range; and

illumination optics, which are configured to project the scanned beam so as to create a pattern of spots extending over a region of interest;

an imaging module, which is configured to capture an image of the pattern that is projected onto an object in the region of interest; and

a processor, which is configured to process the image in order to construct a three-dimensional (3D) map of the object.

2. The apparatus according to claim 1, wherein the pattern of the spots is uncorrelated over a range of depths that is mapped by the apparatus.

3. The apparatus according to claim 1, wherein the radiation source is controlled so as to modulate an intensity of the beam while the scanner scans the beam, thereby creating the pattern of the spots on the region of interest.

4. The apparatus according to claim 3, wherein the illumination module is configured to modify the pattern responsively to the image captured by the imaging module.

5. The apparatus according to claim 4, wherein the illumination module is configured to control at least one of the radiation source and the scanner so as to modify an angular density of the spots in the array within a selected part of the region of interest.

6. The apparatus according to claim 4, wherein the illumination module is configured to control at least one of the radiation source and the scanner so as to modify a brightness of the spots in a selected area with the region of interest.

7. The apparatus according to claim 3, wherein the scanner is configured to scan the beam over a first angular range, and

wherein the optics comprise a beamsplitter, which is configured to create multiple, angularly-spaced replicas of the scanned beam, which together extend over a second angular range, which is greater than the first angular range.

8. The apparatus according to claim 7, wherein the scanner and the beamsplitter are configured to tile the region of interest with the pattern created by the multiple, angularly-spaced replicas of the scanned beam.

9. The apparatus according to claim 1, wherein the optics comprise a patterned element, which is configured, when illuminated by the beam, to create the pattern over a first angular range, and

wherein the scanner is configured to direct the beam to strike the patterned element at multiple different angles in succession so as to create multiple, angularly-spaced replicas of the pattern, which together extend over a second angular range, which is greater than the first angular range.

10. The apparatus according to claim 9, wherein the scanner and the patterned element are configured to tile the region of interest with the multiple, angularly-spaced replicas of the pattern.

11. The apparatus according to claim 1, wherein the scanner is configured to scan the beam over a first angular range, and

wherein the optics comprise a scan-expanding element, which is configured to distribute the scanned beam so as to cover a second angular range, greater than the first angular range, with the spatial pattern.

12. The apparatus according to claim 11, wherein the scan-expanding element is selected from a group of elements consisting of a convex reflector and a diffractive optical element.

13. The apparatus according to claim 1, wherein the illumination module comprises at least one beam sensor, which is positioned at a selected angle within the angular range that is scanned by the scanner so as to receive the scanned beam periodically and verify thereby that the scanner is operating.

14. The apparatus according to claim 13, wherein the illumination module is configured to inhibit emission of the beam from the radiation source when the sensor fails to receive the scanned beam periodically.

15. The apparatus according to claim 1, wherein the radiation source comprises:

a first radiation source, which emits an infrared beam, which is modulated to create the pattern of the spots; and

a second radiation source, which emits a visible light beam, which is modulated to project a visible image onto the region of interest,

wherein the scanner and optics are configured to project both the infrared beam and the visible light beam onto the region of interest simultaneously.

16. The apparatus according to claim 15, wherein the second radiation source is controlled so as to project the visible image onto the object responsively to the 3D map.

17. The apparatus according to claim 1, wherein the processor is arranged to derive the 3D map by finding respective offsets between the spots in areas of the captured image and corresponding reference spot locations belonging to a reference image of the pattern, wherein the respective offsets are indicative of respective distances between the areas and the image capture assembly.

18. The apparatus according to claim 17, wherein the imaging module comprises a position-sensitive detector, which is configured to sense and output an offset of each spot in the pattern on the object as the spot is projected by the illumination module.

19. The apparatus according to claim 18, wherein the imaging module is configured to scan a field of view of the position-sensitive detector in synchronization with the scanner in the illumination module.

20. The apparatus according to claim 18, wherein the scanner is configured to scan a field of view of the position-sensitive detector together with the beam from the radiation source.

21. The apparatus according to claim 17, wherein the illumination module and the imaging module are arranged so that the offsets occur in a first direction, and

wherein the imaging module comprises:

an array of detector elements arranged in one or more rows extending in the first direction; and

astigmatic optics, which are configured to image the pattern onto the array and have a greater optical power in the first direction than in a second, perpendicular direction.

22. The apparatus according to claim 1, wherein the imaging module comprises a sensor and imaging optics, which define a sensing area that is scanned over the region of interest in synchronization with the scanned beam of the illumination module.

23. The apparatus according to claim 22, wherein the sensor comprises an image sensor having a rolling shutter, and wherein the rolling shutter is synchronized with the scanned beam.

24. The apparatus according to claim 22, wherein the scanner in the illumination module is controllable to dynamically vary the selected angular range, and wherein the imaging module comprises an imaging scanner, which is configured to dynamically scan the sensing area to match the selected angular range of the scanned beam.

25. Apparatus for mapping, comprising:

an illumination module, comprising;

a radiation source, which is configured to emit a beam of radiation having an intensity that varies according to a specified temporal modulation; and

a scanner, which is configured to receive and scan the beam over a region of interest, so as to project the radiation onto the region with a spatial intensity pattern determined by the temporal modulation of the beam;

an imaging module, which is configured to capture an image of the spatial intensity pattern that is projected onto an object in the region of interest; and

a processor, which is configured to process the image in order to construct a three-dimensional (3D) map of the object.

26. The apparatus according to claim 25, wherein the temporal modulation is binary, and wherein the spatial intensity pattern comprises an array of spots generated by the temporal modulation.

27. The apparatus according to claim 25, wherein the processor is arranged to derive the 3D map by finding respective offsets between the pattern in areas of the captured image and a reference image of the pattern, wherein the respective offsets are indicative of respective distances between the areas and the image capture assembly.

28. The apparatus according to claim 25, wherein the imaging module comprises a sensor and imaging optics, which define a sensing area that is scanned over the region of interest in synchronization with the scanned beam of the illumination module.

29. A method for mapping, comprising:

scanning a beam of radiation over a selected angular range so as to create a pattern of spots extending over a region of interest;

capturing an image of the pattern that is projected onto an object in the region of interest; and

processing the image in order to construct a three-dimensional (3D) map of the object.

30. The method according to claim 29, wherein the pattern of the spots is uncorrelated over a range of depths that is mapped by the method.

31. The method according to claim 29, wherein scanning the beam comprises modulating an intensity of the beam while the scanning the beam, thereby creating the pattern of the spots on the region of interest.

32. The method according to claim 31, wherein creating the pattern comprises modifying the pattern responsively to the image captured by the imaging module.

33. The method according to claim 32, wherein modifying the pattern comprises modifying an angular density of the spots in the array within a selected part of the region of interest.

34. The method according to claim 32, wherein modifying the pattern comprises modifying a brightness of the spots in a selected area with the region of interest.

35. The method according to claim 31, wherein creating the pattern comprises scanning the beam over a first angular range, and splitting the scanned beam to create multiple, angularly-spaced replicas of the scanned beam, which together extend the pattern over a second angular range, which is greater than the first angular range.

36. The method according to claim 35, wherein splitting the scanned beam comprises tiling the region of interest with the pattern created by the multiple, angularly-spaced replicas of the scanned beam.

37. The method according to claim 29, wherein scanning the beam comprises illuminating a patterned element, which is configured, when illuminated by the beam, to create the pattern over a first angular range, while directing the beam to strike the patterned element at multiple different angles in succession so as to create multiple, angularly-spaced replicas of the pattern, which together extend over a second angular range, which is greater than the first angular range.

38. The method according to claim 37, wherein directing the beam comprises tiling the region of interest with the multiple, angularly-spaced replicas of the pattern.

39. The method according to claim 29, wherein scanning the beam comprises directing the beam, which is scanned over a first angular range, to strike a scan-expanding element, which is configured to distribute the scanned beam so as to cover a second angular range, greater than the first angular range, with the spatial pattern.

40. The method according to claim 39, wherein the scan-expanding element is selected from a group of elements consisting of a convex reflector and a diffractive optical element.

41. The method according to claim 29, and comprising sensing the scanned beam at a selected angle within the angular range, and verifying the scanning by sensing the scanned beam periodically at the selected angle.

42. The method according to claim 41, wherein scanning the beam comprises inhibiting emission of the beam upon a failure to sense the scanned beam periodically.

43. The method according to claim 29, wherein scanning the beam comprises scanning an infrared beam, which is modulated to create the pattern of the spots, and wherein the method comprises scanning a visible light beam together with the infrared beam, while modulating the visible light beam so as to project a visible image onto the region of interest simultaneously with projecting both the infrared beam and the visible light beam onto the region of interest simultaneously.

44. The method according to claim 43, wherein modulating the visible light beam comprises generating the visible image responsively to the 3D map.

45. The method according to claim 29, wherein processing the image comprises deriving the 3D map by finding respective offsets between the spots in areas of the captured image and corresponding reference spot locations belonging to a reference image of the pattern, wherein the respective offsets are indicative of respective distances between the areas and the image capture assembly.

46. The method according to claim 45, wherein capturing the image comprises applying a position-sensitive detector to sense and output an offset of each spot in the pattern on the object as the spot is projected by the illumination module.

47. The method according to claim 46, wherein applying the position-sensitive detector comprises scanning a field of view of the position-sensitive detector in synchronization with the scanner in the illumination module.

48. The method according to claim 46, wherein applying the position-sensitive detector comprises scanning a field of view of the position-sensitive detector together with the beam from the radiation source.

49. The method according to claim 45, wherein the offsets that are indicative of the respective distances occur in a first direction, and wherein capturing the image comprises imaging the pattern onto an array of detector elements arranged in one or more rows extending in the first direction using astigmatic optics having a greater optical power in the first direction than in a second, perpendicular direction.

50. The method according to claim 29, wherein capturing the image comprises scanning a sensing area or a sensor over the region of interest in synchronization with the scanned beam of the illumination module.

51. The method according to claim 50, wherein the sensor comprises an image sensor having a rolling shutter, and wherein scanning the sensing area comprises synchronizing the rolling shutter with the scanned beam.

52. The method according to claim 50, wherein scanning the beam comprises dynamically varying the selected angular range, and scanning the sensing area comprises dynamically scanning the sensing area to match the selected angular range of the scanned beam.

53. A method for mapping, comprising:

generating a beam of radiation having an intensity that varies according to a specified temporal modulation;

scanning the beam over a region of interest, so as to project the radiation onto the region with a spatial intensity pattern determined by the temporal modulation of the beam;

capturing an image of the spatial intensity pattern that is projected onto an object in the region of interest; and

processing the image in order to construct a three-dimensional (3D) map of the object.

54. The method according to claim 53, wherein the temporal modulation is binary, and wherein the spatial intensity pattern comprises an array of spots generated by the temporal modulation.

55. The method according to claim 53, wherein processing the image comprises deriving the 3D map by finding respective offsets between the pattern in areas of the captured image and a reference image of the pattern, wherein the respective offsets are indicative of respective distances between the areas and the image capture assembly.

56. The method according to claim 53, wherein capturing the image comprises scanning a sensing area or a sensor over the region of interest in synchronization with the scanned beam.