Dynamic control over structured illumination intensity

• US 10,349,038 B2
• Filed: 12/27/2016
• Issued: 07/09/2019
• Est. Priority Date: 01/04/2016
• Status: Active Grant

• Current Assignee: META Platforms Technologies LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Original Assignee: Facebook Technologies, LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Inventors: Trail, Nicholas Daniel, McNally, Stephen James
• Primary Examiner(s): Young, Patricia I
• Application Number:
  US15/391,812
• Publication Number:
  US 20170195653A1
• Time in Patent Office: 924 Days
• Field of Search: 348 46
• CPC Class Codes: G02F 1/13306 Circuit arrangements or driving methods for the control of single liquid crystal cells G02F1/132, G02F1/133382 take precedence
• CPC Class Codes: G02F 1/133528 Polarisers
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays
• CPC Class Codes: H04N 13/366 using viewer tracking

Abstract

A depth camera assembly (DCA) determines depth information for a scene in a field of view of the DCA. The DCA includes a structured light (SL) illuminator, a camera, and a controller. The SL illuminator includes a source assembly, a SL element, a liquid crystal (LC) array, and a polarizer. The source assembly generates light, and the SL element generates a SL pattern using the generated light source. The LC array includes a plurality of addressable cells configured to polarize the SL pattern in accordance with adjustment instructions. The polarizer attenuates portions of the SL pattern based on the polarization of the portions of the SL pattern. The camera captures an image of the SL pattern, and the controller identifies portions of the image that are saturated and generates adjustment instructions based in part on the identified portions of the image, and provides the adjustment instructions to the LC array.

Claims

1. A depth camera assembly comprising:

a structured light illuminator configured to illuminate a scene with a variable intensity structured light (SL) pattern, the SL pattern including a plurality of portions and at least one of the portions has a different intensity value than another portion of the plurality of portions, the structured light illuminator comprising;

a source assembly that generates light,

a structured light element that generates a SL pattern using the light,

a liquid crystal array including a plurality of addressable liquid crystal cells, each liquid crystal cell configured to, respectively,

receive a portion of the SL pattern, and

modify a polarization state of the portion of the SL pattern in accordance with adjustment instructions,

a polarizer configured to attenuate portions of the SL pattern based on the polarization of the portions of the SL pattern to generate the variable intensity SL pattern;

a camera configured to capture an image of the scene illuminated with the variable intensity SL pattern; and

a controller coupled to the camera and the structured light illuminator, the controller configured to;

for each of a plurality of portions of the image, determining a saturation factor,

increase an attenuation factor in response to a saturation factor of a portion of the image being greater than a first threshold value,

decrease the attenuation factor in response to the saturation factor of the portion of the image being smaller than a second threshold value, and

control the liquid crystal array based on the attenuation factor.

2. The depth camera assembly of claim 1, wherein the light generated by the source assembly is an infrared light spectrum.

3. The depth camera assembly of claim 1, wherein the liquid crystal array comprises a plurality of independently addressable liquid crystal cells or pixels.

4. The depth camera assembly of claim 1, wherein the controller is further configured to:

determine an attenuation factor based on three dimensional position information of objects located in front of the SL illuminator and the camera, and

control the liquid crystal array based on the determined attenuation factor.

5. The depth camera assembly of claim 1, wherein the controller is further configured to:

increase an attenuation factor in response to a signal-to-noise ratio (SNR) in a portion of the image being greater than a first threshold value;

decrease the attenuation factor in response to the SNR in the portion of the image being smaller than a second threshold value; and

control the liquid crystal array based on the attenuation factor.

6. The depth camera assembly of claim 1, wherein the liquid crystal array is positioned at least 3 millimeters away from the structured light element.

7. The depth camera assembly of claim 1, wherein the structured light element generates a structured light pattern comprising a plurality of dots, and wherein each liquid crystal cell of liquid crystal array corresponds to one dot of the plurality of dots of the structured light pattern.

8. The depth camera assembly of claim 1, wherein the structured light element generates a structured light pattern comprising a plurality of dots, and wherein at least one liquid crystal cell of liquid crystal array corresponds to multiple dots of the plurality of dots of the structured light pattern.

9. The depth camera assembly of claim 1, wherein the depth camera assembly is part of a head-mounted display.

10. A head-mounted display (HMD) comprising:

an electronic display configured to emit image light; and

a depth camera assembly, the depth camera assembly comprising;

a structured light illuminator configured to illuminate a scene with a variable intensity structured light (SL) pattern, the SL pattern including a plurality of portions and at least one of the portions has a different intensity value than another portion of the plurality of portions, the structured light illuminator comprising;

a source assembly that generates light,

a structured light element that generates a SL pattern using the light,

a liquid crystal array including a plurality of addressable liquid crystal cells, each liquid crystal cell configured to, respectively,

receive a portion of the SL pattern, and

modify a polarization state of the portion of the SL pattern in accordance with adjustment instructions,

a polarizer configured to attenuate portions of the SL pattern based on the polarization of the portions of the SL pattern to generate the variable intensity SL pattern;

a camera configured to capture an image of the scene illuminated with the variable intensity SL pattern; and

a controller coupled to the camera and the structured light illuminator, the controller configured to;

for each of a plurality of portions of the image, determining a saturation factor,

increase an attenuation factor in response to a saturation factor of a portion of the image being greater than a first threshold value,

decrease the attenuation factor in response to the saturation factor of the portion of the image being smaller than a second threshold value, and

control the liquid crystal array based on the attenuation factor; and

an optics block configured to direct the image light to an exit pupil of the HMD corresponding to a location of an eye of a user of the HMD.

11. The head-mounted display of claim 10, wherein the liquid crystal array of the depth camera assembly comprises a plurality of independently addressable liquid crystal cells or pixels.

12. The head-mounted display of claim 10, wherein the controller of the depth camera assembly is further configured to:

increase an attenuation factor in response to a signal-to-noise ratio (SNR) in a portion of the image being greater than a first threshold value;

decrease the attenuation factor in response to the SNR in the portion of the image being smaller than a second threshold value; and

control the liquid crystal array based on the attenuation factor.

13. The head-mounted display of claim 10, wherein the liquid crystal array of the depth camera assembly is positioned at least 3 millimeters away from the structured light element.

14. The head-mounted display of claim 10, wherein the structured light element of the depth camera assembly generates a structured light pattern comprising a plurality of dots, and wherein each liquid crystal cell of liquid crystal array corresponds to one dot of the plurality of dots of the structured light pattern.

15. The head-mounted display of claim 10, wherein the structured light element of the depth camera assembly generates a structured light pattern comprising a plurality of dots, and wherein each liquid crystal cell of liquid crystal array corresponds to multiple dots of the plurality of dots of the structured light pattern.

16. A method, comprising:

illuminating a scene with a variable intensity structured light (SL) pattern that is part of a head-mounted display;

capturing an image of the scene illuminated with the variable intensity SL pattern;

identifying portions of the captured image that are saturated;

increasing an attenuation factor of a first portion of the image in response to a signal-to-noise ratio (SNR) in of the first portion of the image being greater than a first threshold value;

decreasing the attenuation factor of a second portion of the image in response to the SNR of the second portion of the image being smaller than a second threshold value; and

attenuating portions of the variable intensity structured light pattern based on the attenuation factor.

17. The method of claim 16, wherein identifying portions of the captured image that are saturated comprises:

identifying portions of the captured image that have a brightness or signal value higher than a threshold value.