Electronic Device with Modulated Light Flash Operation for Rolling Shutter Image Sensor

• US 20150002734A1
• Filed: 11/21/2013
• Published: 01/01/2015
• Est. Priority Date: 07/01/2013
• Status: Abandoned Application

• Current Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Original Assignee: Motorola Mobility LLC (Lenovo Group Ltd.)
• Inventors: Lee, Johnny C
• Application Number:
  US14/086,427
• Publication Number:
  US 20150002734A1
• Time in Patent Office: Days
• US Class Current: 348/367
• CPC Class Codes: G02B 27/425 in illumination systems mask illumination systems in photolithographic systems G03F7/70158
• CPC Class Codes: G03B 9/08 Shutters
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06T 2207/20224 Image subtraction
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G09G 2320/103 Detection of image changes, e.g. determination of an index representative of the image change
• CPC Class Codes: G09G 2360/144 the light being ambient light
• CPC Class Codes: H01S 5/0071 for beam steering, e.g. using a mirror outside the cavity to change the beam direction
• CPC Class Codes: H01S 5/423 having a vertical cavity
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 23/21 from near infrared [NIR] radiation
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/667 Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
• CPC Class Codes: H04N 23/73 : by influencing the exposure time
• CPC Class Codes: H04N 23/74 : by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 25/00 : Circuitry of solid-state image sensors [SSIS]; Control thereof
• CPC Class Codes: H04N 25/53 : Control of the integration time
• CPC Class Codes: H04N 25/531 : by controlling rolling shutters in CMOS SSIS
• CPC Class Codes: H04N 25/705 : Pixels for depth measurement, e.g. RGBZ
• CPC Class Codes: H04N 5/33 : Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

An electronic device (100) includes a modulated light projector (119) and an electronic rolling shutter (ERS) imaging camera (1302) having a sensor array (1306) with a set of pixel rows. The electronic device includes a controller (1304) to control the modulated light projector (119) to project a modulated light flash (1608, 1810, 2010) during capture of a first image frame (1606, 1806, 2006) by the sensor array. The controller is to initiate the modulated light flash while each pixel row of the set is exposed for gathering light and to terminate the modulated light flash before a pixel row of the set is exposed for capture of a next image frame by the sensor array. The electronic device further includes a processor (802) to determine modulated light image data (1320) based on the first image frame. The controller also may control the modulated light projector to refrain from projecting the modulated light flash during capture of a second image frame (1602, 1612, 1802, 1816, 2002, 2018), adjacent to the first image frame.

Claims

1. An electronic device comprising:

a modulated light projector;

an electronic rolling shutter (ERS) imaging camera having a sensor array with a set of pixel rows;

a controller to control the modulated light projector to project a modulated light flash during capture of a first image frame by the sensor array, wherein the controller is to initiate the modulated light flash while each pixel row of the set is exposed for gathering light and to terminate the modulated light flash before a pixel row of the set is exposed for capture of a next image frame by the sensor array; and

a processor to determine modulated light image data based on the first image frame.

2. The electronic device of claim 1, wherein:

the controller is to control the modulated light projector to refrain from projecting a modulated light flash during capture of a second image frame adjacent to the capture of the first image frame.

3. The electronic device of claim 2, wherein:

the ERS imaging camera captures the first image frame and the second image frame at a frame rate at least twice a nominal frame rate of the ERS imaging camera.

4. The electronic device of claim 2, wherein:

the processor is to determine the modulated light image data based on a comparison of the first image frame and the second image frame.

5. The electronic device of claim 1, wherein:

the controller is to control the ERS imaging camera to initiate exposure of the set of pixel rows substantially simultaneously for capture of the first image frame and to initiate the modulated light flash in response to the initiation of the exposure of the set of pixel rows substantially simultaneously.

6. The electronic device of claim 5, further comprising:

a mechanical shutter to terminate exposure of the set of pixel rows substantially simultaneously for capture of the first image frame; and

wherein the controller is to control the modulated light projector to terminate the modulated light flash prior to activation of the mechanical shutter for capture of the first image frame.

7. The electronic device of claim 1, wherein the modulated light projector comprises:

an array of one or more vertical cavity surface emitting laser (VCSEL) diodes;

an array of one or more lenses overlying the array of one or more VCSEL diodes; and

a diffractive optical element overlying the array of one or more lenses.

8. A method comprising:

capturing a first image frame at an electronic rolling shutter (ERS) imaging camera;

initiating a projection of a modulated light flash while each pixel row of a set of pixel rows of a sensor array of the ERS imaging camera is exposed for gathering light for the first image frame;

terminating the modulated light flash before a pixel row of the set is exposed for capture of a next image frame by the ERS imaging camera; and

determining modulated light image data based on the first image frame.

9. The method of claim 8, further comprising:

capturing a second image frame at the ERS imaging camera, the second image frame adjacent to the first image frame; and

refraining from projection of a modulated light flash during the capture of the second image frame.

10. The method of claim 9, wherein:

the first and second image frames are captured at a frame rate at least twice a nominal frame rate of the ERS imaging camera.

11. The method of claim 9, wherein determining the modulated light image data based on the first image frame comprises:

determining the modulated light image data based on a comparison of the first image frame and the second image frame.

12. The method of claim 8, wherein:

capturing the first image frame includes initiating exposure of the set of pixel rows substantially simultaneously; and

initiating the modulated light flash includes initiating the modulated light flash in response to the initiation of the exposure of the set of pixel rows substantially simultaneously.

13. The method of claim 12, wherein:

capturing the first image frame further includes terminating exposure of the set of pixel rows substantially simultaneously; and

terminating the modulated light flash includes terminating the modulated light flash prior to terminating exposure of the set of pixel rows substantially simultaneously.

14. The method of claim 13, wherein terminating exposure of the set of pixel rows substantially simultaneously comprises:

activating a mechanical shutter to terminate exposure of the set of pixels.

15. The method of claim 8, further comprising:

determining at least one spatial feature based on the modulated light image data.

16. A method comprising:

capturing, at an electronic rolling shutter (ERS) imaging camera, a set of image frames at a first frame rate;

capturing, at the ERS imaging camera, a first image frame and a second image frame at a second frame rate, the second frame rate greater than the first frame rate;

projecting a modulated light flash during the capture of the first image frame, a duration of the modulated light flash occurring while each pixel row of the ERS imaging camera is exposed for gathering light for the first image frame;

refraining from projection of a modulated light flash during the capture of the second image frame; and

determining modulated light image data based on a comparison of the first image frame and the second image frame.

17. The method of claim 16, wherein determining the modulated light image data comprises:

subtracting image content of the second image frame from image content of the first image frame.

18. The method of claim 16, further comprising:

displaying image content of the image frames of the set of image frames; and

refraining from displaying image content of the first image frame and the second image frame.

19. The method of claim 16, wherein capturing the first image frame and the second image frame comprises:

capturing the second image frame prior to capturing the first image frame.

20. The method of claim 16, further comprising:

determining at least one spatial feature based on the modulated light image data.