ENVIRONMENT-DEPENDENT ACTIVE ILLUMINATION FOR STEREO MATCHING

• US 20150229915A1
• Filed: 02/08/2014
• Published: 08/13/2015
• Est. Priority Date: 02/08/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Kirk, Adam G., Rhemann, Christoph, Izadi, Shahram, Kang, Sing Bing, Georgiou, Andreas, Whyte, Oliver A.
• Granted Patent: US 11,265,534 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G01B 11/167 by projecting a pattern on the object
• CPC Class Codes: H04N 13/363 using image projection screens volumetric display H04N13/388
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20
• CPC Class Codes: H04N 9/3179 Video signal processing therefor

Abstract

The subject disclosure is directed towards controlling the intensity of illumination of a scene or part of a scene, including to conserve illumination power. Quality of depth data in stereo images may be measured with different illumination states; environmental conditions, such as ambient light, natural texture may affect the quality. The illumination intensity may be controllably varied to obtain sufficient quality while conserving power. The control may be directed to one or more regions of interest corresponding to an entire scene or part of a scene.

Claims

1. A system comprising, a controller, the controller coupled to a projector set that projects a light pattern towards a scene, the projector set comprising one or more projection elements, the controller configured to receive data corresponding to environmental conditions and control the projector set to selectively illuminate at least part of the scene based upon the data.

2. The system of claim 1 wherein the data corresponding to the environmental conditions comprise at least part of stereo image data.

3. The system of claim 2 wherein the data corresponding to the environmental conditions comprises quality data, and further comprising an image processing component configured to process the at least part of the stereo images to obtain a quality measure as the data corresponding to the environmental conditions.

4. The system of claim 3 wherein the quality data corresponds to texture, or variance data, or both texture and variance data.

5. The system of claim 4 wherein the image processing component generates a depth map from at least part of the stereo images, including by combining different parts of the stereo image data based upon the quality measure.

6. The system of claim 1 wherein the controller is configured to turn off and turn on at least one of the one or more projection elements.

7. The system of claim 1 wherein the controller is configured to ramp power up or down, or both up and down, to at least one of the one or more projection elements.

8. The system of claim 1 wherein the projector set comprises at least one infrared projection element, or at least one visible light projection element, or both at least one infrared projection element and at least one visible light projection element.

9. The system of claim 1 wherein the controller is configured to control illumination of at least one region of interest of the scene based upon one or more illumination parameters, in which the region of interest is less than the entire scene capable of being illuminated.

10. The system of claim 1 wherein the controller is configured to independently control each of at least two projection elements.

11. A method comprising, determining depth data based upon stereo images that are dependent at least in part upon illumination of a scene, including obtaining quality data based upon stereo matching using one or more images, and controllably projecting light into the scene based upon the quality data.

12. The method of claim 11 wherein obtaining the quality data comprises illuminating at least part of the scene at a first illumination level at one time and capturing first stereo images during the illuminating with the first illumination level, illuminating the at least part of the scene at a second, different illumination level at another time and capturing second stereo images during the illuminating with the second illumination level, processing at least part of the first stereo images into a first set of data, processing at least part of the second set of stereo images into a second set of data, and combining the first set of data and second set of data into combined quality data.

13. The method of claim 12 further comprising, using the combined quality data to independently control different regions of interest within the scene with independent illumination levels.

14. The method of claim 11 wherein obtaining the quality data comprises illuminating at least part of the scene with visible light and capturing visible light stereo images during the illuminating with the visible light, illuminating the at least part of the scene with infrared light and capturing second stereo images during the illuminating with the infrared light, processing at least part of the visible light stereo images into a first set of data, processing at least part of the infrared stereo images into a second set of data, and combining the first set of data and second set of data.

15. The method of claim 14 further comprising, using the combined quality data to independently illuminate each of at least two different regions of interest within the scene with visible light or infrared light, or both visible light and infrared light.

16. The method of claim 11 further comprising, determining one or more illumination parameters for a region of interest within a scene.

17. The method of claim 11 further comprising, reducing or turning off illumination power to the scene or part of the scene to determine whether illuminating the scene or the part of the scene is beneficial to depth data quality.

18. The method of claim 11 further comprising, increasing or turning on illumination power to the scene or part of the scene to determine whether illuminating the scene or the part of the scene is beneficial to depth data quality.

19. The method of claim 11 further comprising, coordinating the illuminating of the scene with the capture of the one or more images.

20. One or more machine or computer-readable storage media containing executable instructions, which when executed perform steps, comprising, illuminating a region of interest comprising a scene or part of a scene, and processing stereo image data to determine an intensity level for illuminating the region of interest.