Synchronization of projected illumination with rolling shutter of image sensor

• US 20110187878A1
• Filed: 04/19/2010
• Published: 08/04/2011
• Est. Priority Date: 02/02/2010
• Status: Abandoned Application

• Current Assignee: Apple Inc.
• Original Assignee: PrimeSense Ltd (Apple Inc.)
• Inventors: Shpunt, Alexander, Mor, Zafrir
• Application Number:
  US12/762,373
• Publication Number:
  US 20110187878A1
• Time in Patent Office: Days
• US Class Current: 348/218.1
• CPC Class Codes: G02B 19/0028 refractive and reflective surfaces, e.g. non-imaging catadioptric systems
• CPC Class Codes: G02B 19/0057 in the form of a laser diode array, e.g. laser diode bar semiconductor laser arrays with beam combining arrangement H01S5/4012
• CPC Class Codes: G02B 27/0944 Diffractive optical elements, e.g. gratings, holograms gratings per se G02B5/18; holograms used as optical elements per se G02B5/32
• CPC Class Codes: G02B 27/0972 Prisms prisms per se G02B5/04
• CPC Class Codes: G02B 27/0983 being curved
• CPC Class Codes: G03B 15/06 Special arrangements of screening, diffusing, or reflecting devices, e.g. in studio
• CPC Class Codes: H01L 33/60 Reflective elements
• CPC Class Codes: H01S 5/005 Optical components external to the laser cavity, specially adapted therefor, e.g. for homogenisation or merging of the beams or for manipulating laser pulses, e.g. pulse shaping H01S5/026, H01S5/18388 take precedence
• CPC Class Codes: H01S 5/0071 for beam steering, e.g. using a mirror outside the cavity to change the beam direction
• CPC Class Codes: H01S 5/02253 using lenses
• CPC Class Codes: H01S 5/02255 using beam deflecting elements
• CPC Class Codes: H01S 5/02257 using windows, e.g. specially adapted for back-reflecting light to a detector inside the housing
• CPC Class Codes: H01S 5/02315 Support members, e.g. bases or carriers
• CPC Class Codes: H01S 5/02325 Mechanically integrated components on mount members or optical micro-benches
• CPC Class Codes: H01S 5/4031 Edge-emitting structures
• CPC Class Codes: H01S 5/423 having a vertical cavity
• CPC Class Codes: H04N 13/207 using a single 2D image sensor

Abstract

Imaging apparatus includes an illumination assembly, including a plurality of radiation sources and projection optics, which are configured to project radiation from the radiation sources onto different, respective regions of a scene. An imaging assembly includes an image sensor and objective optics configured to form an optical image of the scene on the image sensor, which includes an array of sensor elements arranged in multiple groups, which are triggered by a rolling shutter to capture the radiation from the scene in successive, respective exposure periods from different, respective areas of the scene so as to form an electronic image of the scene. A controller is coupled to actuate the radiation sources sequentially in a pulsed mode so that the illumination assembly illuminates the different, respective areas of the scene in synchronization with the rolling shutter.

Claims

1. Imaging apparatus, comprising:

an illumination assembly, comprising a plurality of radiation sources and projection optics, which are configured to project radiation from the radiation sources onto different, respective regions of a scene;

an imaging assembly, comprising an image sensor and objective optics configured to form an optical image of the scene on the image sensor, which comprises an array of sensor elements arranged in multiple groups, which are triggered by a rolling shutter to capture the radiation from the scene in successive, respective exposure periods from different, respective areas of the scene so as to form an electronic image of the scene; and

a controller, which is coupled to actuate the radiation sources sequentially in a pulsed mode so that the illumination assembly illuminates the different, respective areas of the scene in synchronization with the rolling shutter.

2. The apparatus according to claim 1, wherein each group comprises one or more rows of the sensor elements, and wherein the regions define stripes that extend across the scene in a direction parallel to the rows.

3. The apparatus according to claim 2, wherein each stripe illuminates a respective region that contains the areas of the scene from which the sensor elements in a respective set of multiple rows capture the radiation.

4. The apparatus according to claim 2, wherein the controller is configured to actuate the radiation sources so that the projected radiation sweeps across the scene in a direction perpendicular to the rows.

5. The apparatus according to claim 1, wherein the rolling shutter defines a frame time for capturing the entire electronic image, and wherein the controller is configured to actuate each of the radiation sources for a respective actuation period that is less than half the frame time.

6. The apparatus according to claim 5, wherein the controller is configured to actuate each of the radiation sources so that the illumination assembly illuminates each area of the scene only during a respective exposure period of a corresponding group of the sensor elements that captures the radiation from the area.

7. The apparatus according to claim 1, wherein the projection optics comprise a patterning element, which is configured so that the radiation is projected onto the scene in a predefined pattern, which is detectable in the electronic image formed by the imaging assembly.

8. The apparatus according to claim 7, wherein the controller is configured to analyze the pattern in the electronic image so as to generate a depth map of the scene.

9. The apparatus according to claim 7, wherein the radiation sources comprise a matrix of light-emitting elements, which are arranged on a substrate and are configured to emit the radiation in a direction perpendicular to the substrate.

10. The apparatus according to claim 7, wherein the radiation sources comprise a row of edge-emitting elements, which are arranged on a substrate and are configured to emit the radiation in a direction parallel to the substrate.

11. The apparatus according to claim 10, wherein the illumination assembly comprises a reflector disposed on the substrate so as to turn the radiation emitted by the edge-emitting elements away from the substrate and toward the patterning element.

12. A method for imaging, comprising:

arranging a plurality of radiation sources to project radiation onto different, respective regions of the scene;

configuring an image sensor, which comprises an array of sensor elements arranged in multiple groups, to receive an optical image of the scene, in which the groups of the sensor elements receive the radiation from different, respective areas of the scene;

triggering the groups of the sensor elements with a rolling shutter to capture the radiation from the scene in successive, respective exposure periods so as to form an electronic image of the scene; and

actuating the radiation sources sequentially in a pulsed mode so as to illuminate the different, respective areas of the scene in synchronization with the rolling shutter.

13. The method according to claim 12, wherein each group comprises one or more rows of the sensor elements, and wherein the regions define stripes that extend across the scene in a direction parallel to the rows.

14. The method according to claim 13, wherein each stripe illuminates a respective region that contains the areas of the scene from which the sensor elements in a respective set of multiple rows capture the radiation.

15. The method according to claim 13, wherein actuating the radiation sources causes the projected radiation to sweep across the scene in a direction perpendicular to the rows.

16. The method according to claim 12, wherein the rolling shutter defines a frame time for capturing the entire electronic image, and wherein actuating the radiation sources comprises actuating each of the radiation sources for a respective actuation period that is less than half the frame time.

17. The method according to claim 16, wherein actuating each of the radiation sources comprises illuminating each area of the scene only during a respective exposure period of a corresponding group of the sensor elements that captures the radiation from the area.

18. The method according to claim 12, wherein arranging the plurality of the radiation source comprises projecting the radiation onto the scene in a predefined pattern, which is detectable in the electronic image.

19. The method according to claim 18, and comprising analyzing the pattern in the electronic image so as to generate a depth map of the scene.

20. The method according to claim 18, wherein the radiation sources comprise a matrix of light-emitting elements, which are arranged on a substrate and are configured to emit the radiation in a direction perpendicular to the substrate.

21. The method according to claim 18, wherein the radiation sources comprise a row of edge-emitting elements, which are arranged on a substrate and are configured to emit the radiation in a direction parallel to the substrate.

22. The method according to claim 21, wherein arranging the plurality of the radiation sources comprises providing a reflector on the substrate so as to turn the radiation emitted by the edge-emitting elements away from the substrate and toward the scene.

23. The method according to claim 12, wherein configuring the image sensor comprises arranging multiple image sensors, having respective rolling shutters, together with multiple, respective pluralities of the radiation sources to form respective electronic images of different, respective, overlapping parts of a scene, and

wherein actuating the radiation sources comprises synchronizing the respective pluralities of the radiation sources over the multiple image sensors so as to control an overlap of the respective areas of the scene illuminated by the radiation sources at any given time.

24. The method according to claim 23, and comprising analyzing the pattern over the electronic images formed by the multiple image sensors in order to generate a depth map of the scene.

25. Imaging apparatus, comprising:

multiple imaging units, comprising;

respective pluralities of radiation sources and projection optics, which are configured to project radiation from the radiation sources onto different, respective regions of a scene; and

respective imaging assemblies, comprising respective image sensors and objective optics configured to form respective optical images of different, respective, overlapping parts of the scene on the respective image sensors, each image sensor comprising an array of sensor elements arranged in multiple groups, which are triggered by a rolling shutter to capture the radiation from the scene in successive, respective exposure periods from different, respective areas of the scene so as to form respective electronic images of the scene,

wherein the radiation sources are actuated sequentially in a pulsed mode so that the illumination assembly illuminates the different, respective areas of the scene in synchronization with the rolling shutter, while synchronizing the respective pluralities of the radiation sources over the multiple image sensors so as to control an overlap of the respective areas of the scene illuminated by the radiation sources at any given time.

26. The apparatus according to claim 25, and comprising a controller, which is configured to analyze the pattern over the electronic images formed by the image sensors in the multiple imaging units so as to generate a depth map of the scene.

27. The apparatus according to claim 25, wherein the overlap is controlled so that the respective areas of the scene illuminated by the radiation sources at any given time are non-overlapping.