Camera with tunable filter and active illumination

• US 10,477,173 B1
• Filed: 05/23/2018
• Issued: 11/12/2019
• Est. Priority Date: 05/23/2018
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Ortiz Egea, Sergio, Akkaya, Onur Can, Bamji, Cyrus Soli
• Primary Examiner(s): Owens, Tsion B
• Application Number:
  US15/987,852
• Publication Number:
  US 20190364254A1
• Time in Patent Office: 538 Days
• CPC Class Codes: G01S 17/36 with phase comparison between the received signal and the contemporaneously transmitted signal
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 7/4815 using multiple transmitters
• CPC Class Codes: G01S 7/4816 of receivers alone
• CPC Class Codes: G02B 5/201 in the form of arrays
• CPC Class Codes: G02F 1/0136 for the control of polarisation, e.g. state of polarisation [SOP] control, polarisation scrambling, TE-TM mode conversion or separation G02F1/0353 takes precedence
• CPC Class Codes: G02F 1/133528 Polarisers
• CPC Class Codes: G02F 1/133541 Circular polarisers
• CPC Class Codes: G02F 1/13473 for wavelength filtering or for colour display without the use of colour mosaic filters
• CPC Class Codes: G02F 1/13478 based on selective reflection
• CPC Class Codes: G02F 1/13718 based on a change of the texture state of a cholesteric liquid crystal
• CPC Class Codes: G02F 1/19 based on variable-reflection or variable-refraction elements not provided for in groups G02F1/015 - G02F1/169
• CPC Class Codes: G02F 2201/346 distributed (Bragg) reflector
• CPC Class Codes: G02F 2203/055 wavelength filtering
• CPC Class Codes: G02F 2203/11 involving infrared radiation
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: H04N 2209/043 using an alternating colour separation filter, e.g. colour wheel or colour LCD
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/50 Constructional details
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/64 : Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 5/33 : Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20
• CPC Class Codes: H04N 9/68 : for controlling the amplitude of colour signals, e.g. automatic chroma control circuits H04N9/71, H04N9/73 take precedence; camera processing pipelines for controlling the colour saturation of colour signals H04N23/86

Abstract

A camera includes an optical filter for a sensor array of the camera. The optical filter includes a plurality of liquid crystal layers switchable between a reflection state and a transmission state. Each liquid crystal layer is configured to block spectral light in a different spectral light sub-band and transmit spectral light outside of the spectral light sub-band in the reflection state, and to transmit spectral light in the spectral light sub-band in the transmission state. An active illuminator of the camera is configured to emit active light in a selected spectral light sub-band. One or more of the plurality of liquid crystal layers is switched from the transmission state to the reflection state to tune the optical filter to block spectral light in all but the selected spectral light sub-band. The sensors of the sensor array are addressed to measure spectral light in the selected spectral light sub-band.

Claims

1. A camera comprising:

a sensor array including a plurality of sensors;

an optical filter for the sensor array including a plurality of liquid crystal layers switchable between a reflection state and a transmission state, each liquid crystal layer configured to block spectral light in a different spectral light sub-band and transmit spectral light outside of the spectral light sub-band in the reflection state, and to transmit spectral light in the spectral light sub-band in the transmission state;

a spectral controller machine configured to;

switch one or more liquid crystal layers of the plurality of liquid crystal layers from the transmission state to the reflection state to tune the optical filter to block spectral light in all but a selected spectral light sub-band, and

address the sensors of the sensor array to measure spectral light in the selected spectral light sub-band; and

an output machine operatively connected to the sensor array and configured to output a matrix of pixels based on measurements of the sensor array, each pixel of the matrix including a spectral value for the selected spectral light sub-band.

2. The camera of claim 1, wherein each liquid crystal layer of the plurality of liquid crystal layers includes a plurality of liquid crystals configured to form cholesteric phase structures that block light in a spectral light sub-band and transmit light outside of the spectral light sub-band in the reflection state, and form a nematic phase arrangement that transmits light in the spectral light sub-band in the transmission state.

3. The camera of claim 1, wherein the optical filter is configured to transmit infrared (IR) light in an IR light sub-band in both the transmission state and the reflection state.

4. The camera of claim 1, wherein the optical filter is configured to block substantially all visible light in all visible light sub-bands when all of the plurality of liquid crystal layers are in the reflection state.

5. The camera of claim 1, wherein the spectral controller machine is configured to adjust a center wavelength of the selected spectral light sub-band by switching one or more liquid crystal layers from the transmission state to the reflection state.

6. The camera of claim 1, wherein the spectral controller machine is configured to adjust a bandwidth of the selected spectral light sub-band by switching one or more liquid crystal layers from the transmission state to the reflection state.

7. The camera of claim 1, wherein one or more of the plurality of liquid crystal layers includes a first liquid crystal layer configured to block right-handed circularly polarized light in a spectral light sub-band and transmit light outside of the spectral light sub-band in the reflection state, and a second liquid crystal layer configured to block left-handed circularly polarized light in the spectral light sub-band and transmit light outside of the spectral light sub-band.

8. The camera of claim 1, wherein the spectral controller machine is configured to tune the optical filter for each of a plurality of different spectral light sub-bands and identify an ambient illuminant based on sensor measurements corresponding to the plurality of different spectral light sub-bands.

9. The camera of claim 8, wherein the spectral value for the selected spectral light sub-band is adjusted based on the ambient illuminant.

10. The camera of claim 1, further comprising:

an IR illuminator configured to emit active IR light in an IR light sub-band; and

a depth controller machine configured to;

switch the plurality of liquid crystal layers of the optical filter to the reflection state,

activate the IR illuminator to illuminate a subject with the active IR light,

address each of the sensors of the sensor array to measure an aspect of the active IR light emitted from the IR illuminator and reflected from the subject back to each of the sensors, and

determine, for each of the sensors, a depth value indicative of a depth of the subject based on the measured aspect of the active IR light, and

wherein each pixel of the matrix includes a depth value, and wherein each spectral value is calculated based on the depth value estimated for the sensor corresponding to the pixel.

11. The camera of claim 1, further comprising:

a spectral illuminator configured to emit active spectral light;

wherein the spectral controller machine is configured to, for each of a plurality of spectral light sub-bands;

tune the optical filter to block spectral light outside of the spectral light sub-band and allow transmission of spectral light in the spectral light sub-band,

activate the spectral illuminator to illuminate a subject with active spectral light, and

address each of the sensors of the sensor array to measure active spectral light in the spectral light sub-band emitted from the spectral illuminator and reflected from the subject back to each of the sensors, and

wherein each pixel of the matrix includes a plurality of spectral values, each spectral value corresponding to a spectral light sub-band of the plurality of spectral light sub-bands.

12. The camera of claim 1, further comprising:

a plurality of spectral illuminators, each spectral illuminator configured to emit active spectral light in a different spectral light sub-band;

wherein the spectral controller machine is configured to, for each of the plurality of spectral illuminators;

tune the optical filter to block spectral light outside of the spectral light sub-band of the spectral illuminator and allow transmission of spectral light in the spectral light sub-band of the spectral illuminator,

activate the spectral illuminator to illuminate a subject with active spectral light in the spectral light sub-band of the spectral illuminator, and

address each of the sensors of the sensor array to measure the active spectral light in the spectral light sub-band emitted from the spectral illuminator and reflected from the subject back to each of the sensors, and

wherein each pixel of the matrix includes a plurality of spectral values, each spectral value corresponding to a spectral light sub-band of one of the plurality of spectral illuminators.

13. The camera of claim 1, further comprising:

a spectral illuminator configured to emit non-modulated active spectral light; and

wherein the spectral controller machine is configured to;

activate the spectral illuminator to illuminate a subject with non-modulated active spectral light,

for each of a plurality of spectral light sub-bands,

tune the optical filter to block spectral light that is outside of the spectral light sub-band and allow transmission of spectral light in the spectral light sub-band, and

address each of the sensors of the sensor array to measure non-modulated active spectral light in the spectral light sub-band emitted from the spectral illuminator and reflected from the subject back to each of the sensors,

de-activate the spectral illuminator,

for each of a plurality of spectral light sub-bands,

tune the optical filter to block spectral light that is outside of the spectral light sub-band and allow transmission of spectral light in the spectral light sub-band, and

address each of the sensors of the sensor array to measure ambient light in the spectral light sub-band reflected from the subject back to each of the sensors, and

wherein each pixel of the matrix includes a plurality of spectral values, each spectral value corresponding to a different one of the plurality of spectral light sub-bands and wherein each spectral value is determined based on a differential measurement of the spectral light in the spectral light sub-band when the spectral illuminator is activated and when the spectral illuminator is de-activated.

14. An image acquisition method, the method comprising:

switching one or more liquid crystal layers of a plurality of switchable liquid crystal layers of an optical filter of a camera from a transmission state to a reflection state to tune the optical filter to block light in all but a selected spectral light sub-band, each liquid crystal layer of the plurality of liquid crystal layers configured to block spectral light in a different spectral light sub-band and transmit spectral light outside of the spectral light sub-band in the reflection state, and to transmit spectral light in the spectral light sub-band in the transmission state;

addressing each of a plurality of sensors of a sensor array of the camera to measure spectral light in the selected spectral light sub-band; and

outputting a matrix of pixels based on measurements of the sensor array, each pixel of the matrix including a spectral value for the selected spectral light sub-band.

15. The method of claim 14, further comprising:

switching the plurality of liquid crystal layers of the optical filter to the reflection state;

activating an IR illuminator of the camera to illuminate a subject with active IR light in an IR light sub-band;

addressing each of a plurality of sensors of the sensor array to measure an aspect of the active IR light emitted from the IR illuminator and reflected from the subject back to each of the sensors;

determining a depth value for each of the sensors based on the measured aspect of the active IR light; and

wherein each pixel of the matrix includes a depth value, and wherein each spectral value is calculated based on the depth value determined for the sensor corresponding to the pixel.

16. The method of claim 14, further comprising:

for each of a plurality of spectral illuminators of the camera,

tuning the optical filter to block spectral light outside of the spectral light sub-band of the spectral illuminator and allow transmission of spectral light in the spectral light sub-band of the spectral illuminator;

activating the spectral illuminator to illuminate the subject with active spectral light in the spectral light sub-band of the spectral illuminator; and

addressing each of the sensors of the sensor array to measure the active spectral light in the spectral light sub-band emitted from the spectral illuminator and reflected from the subject back to each of the sensors;

wherein each pixel of the matrix includes a plurality of spectral values, each spectral value corresponding to a spectral light sub-band of one of the plurality of spectral illuminators.

17. The method of claim 14, wherein each liquid crystal layer of the plurality of liquid crystal layers includes a plurality of liquid crystals configured to form cholesteric phase structures that block light in a spectral light sub-band and transmit light outside of the spectral light sub-band in the reflection state, and form a nematic phase arrangement that transmits light in the spectral light sub-band in the transmission state.

18. The method of claim 14, wherein the one or more liquid crystal layers of the optical filter are switched to adjust one or more of a center wavelength of the selected spectral light sub-band and a bandwidth of the selected spectral light sub-band.

19. The method of claim 14, wherein one or more of the plurality of liquid crystal layers includes a first liquid crystal layer configured to block right-handed circularly polarized light in a spectral light sub-band and transmit spectral light outside of the spectral light sub-band in the reflection state, and a second liquid crystal layer configured to block left-handed circularly polarized light in the spectral light sub-band and transmit spectral light outside of the spectral light sub-band.

20. A camera comprising:

a sensor array including a plurality of sensors;

an IR illuminator configured to emit active IR light in an IR light sub-band;

a plurality of spectral illuminators, each spectral illuminator configured to emit active spectral light in a different spectral light sub-band;

an optical filter for the sensor array including a plurality of liquid crystal layers individually switchable between a reflection state and a transmission state, each liquid crystal layer configured to block spectral light in a different spectral light sub-band and transmit spectral light outside of the spectral light sub-band in the reflection state, and to transmit spectral light in the spectral light sub-band along with spectral light outside the spectral light sub-band in the transmission state;

a depth controller machine configured to;

switch the plurality of liquid crystal layers of the optical filter to the reflection state,

activate the IR illuminator to illuminate a subject with the active IR light,

address each of the sensors of the sensor array to measure an aspect of the active IR light emitted from the IR illuminator and reflected from the subject back to each of the sensors, and

determine, for each of the sensors, a depth value indicative of a depth of the subject based on the measured aspect of the active IR light, and

a spectral controller machine is configured to, for each of the plurality of spectral illuminators;

switch one or more of the plurality of liquid crystal layers of the optical filter from the transmission state to the reflection state to tune the optical filter to block spectral light outside of the spectral light sub-band of the spectral illuminator and allow transmission of spectral light in the spectral light sub-band of the spectral illuminator,

activate the spectral illuminator to illuminate the subject with active spectral light in the spectral light sub-band of the spectral illuminator, and

address each of the sensors of the sensor array to measure the active spectral light in the spectral light sub-band emitted from the spectral illuminator and reflected from the subject back to each of the sensors, and

an output machine configured to output a matrix of pixels, each pixel corresponding to one of the plurality of sensors and including a depth value and a spectral value for each spectral light sub-band, each spectral value calculated based on the depth value estimated for the sensor corresponding to the pixel.