Rapid synchronized lighting and shuttering

• US 9,544,504 B2
• Filed: 10/23/2015
• Issued: 01/10/2017
• Est. Priority Date: 11/02/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Rihn, Bernard K., Dietz, Paul Henry, Keam, Nigel Stuart, Bathiche, Steven Nabil
• Primary Examiner(s): OSINSKI, MICHAEL S
• Application Number:
  US14/921,569
• Publication Number:
  US 20160044225A1
• Time in Patent Office: 445 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

This document describes various apparatuses and techniques for rapid synchronized lighting and shuttering. These apparatuses and techniques are capable of capturing two images, where one of the images integrates multiple exposures during which an image area is illuminated by a light source and another of the images integrates multiple exposures during which the image is not illuminated by the light source. The image area can be illuminated by rapidly flashing the image area with the light source and synchronizing a shutter to permit one image sensor to capture an image when the image area is illuminated and another image sensor to capture the image area when the image area is not illuminated. These two images can be captured concurrently or nearly concurrently, thereby reducing or eliminating motion artifacts. Further, these apparatuses and techniques may do so with slow and relatively low-cost cameras and relatively low computational costs.

Claims

1. A system comprising:

a controller configured to;

synchronize flashing of a first light source and a second light source and shuttering of a shutter effective to capture a first image integrating multiple exposures during which the first light source is flashed and a second image integrating multiple exposures during which the second light source is flashed; and

provide the first image and the second image, the first image and the second image are processed to determine a position or location of an object captured in the first image and the second image.

2. A system as recited in claim 1, wherein the first light source illuminates the object from a different direction than the second light source.

3. A system as recited in claim 2, wherein the first image and the second image are processed to determine a position or location of an object captured in the first image and the second image by comparing a first shadow of the object as illuminated by the first light source in the first image and a second shadow of the object as illuminated by the second light source in the second image.

4. A system as recited in claim 3, wherein the object is moving and its location is tracked through the movement based on shadows of the object as illuminated by the first light source and the second light source.

5. A system as recited in claim 3, wherein the object is a finger, hand, arm, body, or stylus performing a gesture and the position or location of the object is processed for three-dimensional gesture recognition.

6. A system as recited in claim 1, further configured to:

capture a third image through the synchronized shuttering in which the object is not illuminated by the first light source or the second light source; and

provide the third image with the first image and the second image.

7. A system as recited in claim 1, wherein the first light source and the second light source are LEDs or laser diodes and a flash rate of the LEDs or laser diodes is at least 60 Hz.

8. A system as recited in claim 7, wherein the flash rate of the LEDs or laser diodes is at least twice as fast as an image-capture rate at which the first and second images are captured.

9. A system as recited in claim 8, wherein the controller is further configured to synchronize the shuttering of the shutter at a shuttering rate at least as fast as the flash rate.

10. A system as recited in claim 1, wherein the shutter includes a beam splitter, a polarizer, and a ferro-electric polarization retarder and the controller is further configured to control the beam splitter, the polarizer, and the ferro-electric polarization retarder as part of synchronizing the flashing of the first light source and the second light source and the shuttering of the shutter.

11. A system as recited in claim 1, wherein the controller is configured to synchronize flashing of the first light source and the second light source and shuttering of the shutter effective to capture of the first image by a first image sensor and capture of the second image by a second image sensor.

12. A method comprising:

rapidly flashing an object with a first light source and a second light source;

synchronizing shuttering of a first shutter to the rapid flashing of the first light source effective to enable capture a first image integrating multiple exposures;

synchronizing shuttering of a second shutter to the rapid flashing of the second light source effective to enable capture of a second image integrating multiple exposures;

comparing the first image and the second image effective to determine a position or location of an object captured in the first image and the second image.

13. A method as described in claim 12, wherein the first light source illuminates the object from a different angle than the second light source.

14. A method as described in claim 13, wherein comparing the first image and the second image effective to determine a position or location of the object compares a first shadow of the object as illuminated by the first light source in the first image and a second shadow of the object as illuminated by the second light source in the second image.

15. A method as described in claim 14, wherein the object is moving and its location is tracked through the movement based on shadows of the object as illuminated by the first light source and the second light source.

16. A method as described in claim 12, wherein the synchronizing shuttering of the first shutter synchronizes the shuttering of the first shutter at a rate at least as fast as a rate of the rapid flashing of the first light source and the synchronizing shuttering of the second shutter synchronizes the shuttering of the second shutter at a rate at least as fast as a rate of the rapid flashing of the second light source.

17. A computing device comprising:

a first light source configured to flash at a first flash rate;

a second light source configured to flash at a second flash rate;

a first mechanical shutter configured to shutter at least as fast as the first flash rate;

a second mechanical shutter configured to shutter at least as fast as the second flash rate;

a controller configured to;

synchronize flashing of the first light source and shuttering of the first shutter effective to capture a first image integrating multiple exposures during which the first light source is flashed and synchronizing flashing of the second light source and shuttering of the second shutter effective to enable capture of a second image integrating multiple exposures during which the second light source is flashed; and

provide the first image and the second image, the first image and the second image are processed to determine a position or location of an object captured in the first image and the second image.

18. A computing device as described in claim 17, wherein the first light source illuminates the object from a different direction than the second light source.

19. A computing device as described in claim 18, wherein the first image and the second image are processed to determine a position or location of the object by comparing a first shadow of the object as illuminated by the first light source in the first image and a second shadow of the object as illuminated by the second light source in the second image.

20. A computing device as described in claim 19, wherein the object is moving and its location is tracked through the movement based on shadows of the object as illuminated by the first light source and the second light source.