INTEGRATED LOW POWER DEPTH CAMERA AND PROJECTION DEVICE

• US 20120075534A1
• Filed: 09/28/2010
• Published: 03/29/2012
• Est. Priority Date: 09/28/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Katz, Sagi, Adler, Avishai, Yahav, Giora, Felzenshtein, Shlomo
• Granted Patent: US 8,681,255 B2
• Time in Patent Office: Days
• US Class Current: 348/602
• CPC Class Codes: G01S 17/18 wherein range gates are used
• CPC Class Codes: G01S 17/894 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
• CPC Class Codes: H04N 9/3182 Colour adjustment, e.g. white balance, shading or gamut white balance per se H04N9/73; control of amplitude of colour signals H04N9/68; colour control circuits for displays in general G09G5/02, G09G3/2003, G09G3/3607
• CPC Class Codes: H04N 9/3194 including sensor feedback

Abstract

A video projector device includes a visible light projector to project an image on a surface or object, and a visible light sensor, which can be used to obtain depth data regarding the object using a time-of-flight principle. The sensor can be a charge-coupled device which obtains color images as well as obtaining depth data. The projected light can be provided in successive frames. A frame can include a gated sub-frame of pulsed light followed by continuous light, while the sensor is gated, to obtain time of flight data, an ungated sub-frame of pulsed light followed by continuous light, while the sensor is ungated, to obtain reflectivity data and a background sub-frame of no light followed by continuous light, while the sensor is gated, to determine a level of background light. A color sub-frame projects continuous light, while the sensor is active.

Claims

1. A video projector device, comprising:

a light source which emits visible light;

a driver which drives the light source;

at least one optical component which projects the visible light in a field of view;

a sensor which senses visible light, including visible light which is reflected from at least one object in the field of view, the sensor comprises a plurality of pixels; and

at least one control circuit which provides a gated sub-frame in which the light source is driven in a pulsed mode while the sensor is operated in a gated mode, followed by the light source being driven in a continuous mode while the sensor is read to obtain light intensity values, where the at least one control circuit derives depth data regarding the at least one object in the field of view based on the light intensity values, using a time-of-flight principle.

2. The video projector device of claim 1, wherein:

the at least one control circuit provides an ungated sub-frame in which the light source is driven in the pulsed mode while the sensor is operated in an ungated mode, followed by the light source being driven in the continuous mode while the sensor is read to obtain light intensity values, where the at least one control circuit adjusts the light intensity values obtained during the gated sub-frame based on the light intensity values obtained during the ungated sub-frame, to compensate for reflectivity of the at least one object in the field of view.

3. The video projector device of claim 1, wherein:

the at least one control circuit provides at least one background sub-frame in which no visible light is emitted while the sensor is operated in a gated mode, followed by the light source being driven in the continuous mode while the sensor is read to obtain light intensity values, where the at least one control circuit adjusts the light intensity values obtained during the gated sub-frame based on the light intensity values obtained during the background sub-frame, to compensate for background light in the field of view.

4. The video projector device of claim 3, wherein:

the at least one control circuit provides two of the background sub-frames in a frame.

5. The video projector device of claim 1, wherein:

the at least one control circuit provides a color sub-frame in which the light source is driven in the continuous mode while the sensor is operated in a standard color sensing mode, followed by the light source being driven in the continuous mode while the sensor is read to obtain light intensity values which include color data from the field of view.

6. The video projector device of claim 1, wherein:

the at least one control circuit provides a sub-frame in which the light source is driven in the continuous mode while the sensor is active.

7. The video projector device of claim 1, wherein:

the at least one control circuit drives the light source using triangular shaped pulses during the gated sub-frame.

8. The video projector device of claim 1, wherein:

the at least one control circuit drives the light source using pulses which each have at least one of;

(a) a leading edge which transitions higher at a rate which is below a step up rate of the light source, and (b) a trailing edge which transitions lower at a rate which is below a step down rate of the light source.

9. The video projector device of claim 1, wherein:

the light source is a backlight of the video projector device.

10. The video projector device of claim 1, wherein:

the light source, the light source, at least one optical component, the at least one control circuit and the sensor are provided in a common housing.

11. The video projector device of claim 1, further comprising:

at least one light-transmissive LCD panel which encodes video information onto the visible light, the at least one light-transmissive LCD panel comprises a plurality of pixels.

12. The video projector device of claim 1, wherein:

the at least one control circuit focuses the at least one optical component based on the depth data.

13. A video projector device, comprising:

a light source which emits visible light;

a driver which drives the light source;

at least one optical component which projects the modulated visible light in a field of view;

a sensor which senses visible light, including visible light which is reflected from at least one object in the field of view, the sensor comprises a plurality of pixels; and

at least one control circuit which provides a gated sub-frame in which the light source is driven in a pulsed mode so that pulsed visible light is emitted while the sensor is operated in a gated mode, followed by no visible light being emitted while the sensor is read to obtain light intensity values, where the at least one control circuit derives depth data regarding the at least one object in the field of view based on the light intensity values, using a time-of-flight principle.

14. The video projector device of claim 13, wherein:

the at least one control circuit provides an ungated sub-frame in which the light source is driven in the pulsed mode so that pulsed visible light is emitted while the sensor is operated in an ungated mode, followed by no visible light being emitted while the sensor is read to obtain light intensity values, where the at least one control circuit normalizes the light intensity values obtained during the gated sub-frame based on the light intensity values obtained during the ungated sub-frame, to compensate for reflectivity of the at least one object in the field of view.

15. The video projector device of claim 13, wherein:

no visible light is emitted at a time other than during the gated sub-frame and the ungated sub-frame.

16. The video projector device of claim 15, wherein:

the light source is turned off to cause no visible light to be emitted.

18. The video projector device of claim 16, wherein:

the at least one control circuit causes the light source to emit visible light at a higher illumination power during the depth-sensing frame than during the non-depth-sensing frame.

19. The video projector device of claim 16, wherein:

the at least one control circuit adjusts a frame period so that the depth-sensing frame is longer than the non-depth-sensing frame.

20. The video projector device of claim 16, wherein:

the at least one control circuit is controllable to selectively operate in a first mode in which the at least one control circuit uses a limited number of one or more frames to obtain depth data regarding the at least one object in the field of view, and in a second mode in which the at least one control circuit continuously obtains depth data regarding at least one object in the field of view.

17. A video projector device, comprising:

a light source which emits visible light;

a driver which drives the light source;

at least one optical component which projects the visible light in a field of view;

a sensor which senses visible light, including visible light which is reflected from at least one object in the field of view, the sensor comprises a plurality of pixels; and

at least one control circuit which provides, at different times;

(a) a depth-sensing frame in which the at least one control circuit derives depth data regarding the at least one object in the field of view based on light intensity values which are read from the sensor, using a time-of-flight principle, and (b) a non-depth-sensing frame in which the at least one control circuit does not derive depth data regarding the at least one object in the field of view.