Context-based depth sensor control

• US 10,038,893 B2
• Filed: 06/15/2016
• Issued: 07/31/2018
• Est. Priority Date: 02/28/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Lee, Johnny
• Primary Examiner(s): Tran, Thai
• Assistant Examiner(s): Mesa, Jose
• Application Number:
  US15/182,963
• Publication Number:
  US 20170019658A1
• Time in Patent Office: 776 Days
• Field of Search: 348 47
• CPC Class Codes: G01B 11/2513 with several lines being projected in more than one direction, e.g. grids, patterns
• CPC Class Codes: G01B 11/2545 with one projection direction and several detection directions, e.g. stereo
• CPC Class Codes: G01C 3/08 Use of electric radiation detectors
• CPC Class Codes: G01S 17/08 for measuring distance only indirect measurement G01S17/46; active triangulation systems G01S17/48
• CPC Class Codes: G01S 17/46 Indirect determination of position data
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 25/00 Circuitry of solid-state image sensors [SSIS]; Control thereof
• CPC Class Codes: H04N 25/705 Pixels for depth measurement, e.g. RGBZ

Abstract

An electronic device (100) includes a depth sensor (120), a first imaging camera (114, 116), and a controller (802). The depth sensor (120) includes a modulated light projector (119) to project a modulated light pattern (500). The first imaging camera (114, 116) is to capture at least a reflection of the modulated light pattern (500). The controller (802) is to selectively modify (1004) at least one of a frequency, an intensity, and a duration of projections of the modulated light pattern by the modulated light projector responsive to at least one trigger event (1002). The trigger event can include, for example, a change (1092) in ambient light incident on the electronic device, detection (1094) of motion of the electronic device, or a determination (1096) that the electronic device has encountered a previously-unencountered environment.

Claims

1. An electronic device comprising:

a first imaging camera;

a second imaging camera;

a modulated light projector to project at least a modulated light pattern;

an ambient light sensor to detect an ambient light condition of the electronic device;

a depth sensor including the modulated light projector and at least one of the first imaging camera and the second imaging camera;

a controller to control at least one of a frequency, an intensity, and a duration of projections of the modulated light pattern by the modulated light projector responsive to the ambient light condition;

wherein the electronic device is to determine depth data for detected spatial features using stereoscopic analysis of image data from the first imaging camera and image data from the second imaging camera responsive to the ambient light condition being greater than a threshold; and

wherein the electronic device is to determine depth data for detected spatial features using reflections of the modulated light pattern captured by one of the first imaging camera or the second imaging camera responsive to the ambient light condition being less than the threshold.

2. The electronic device of claim 1, wherein the controller is to a control the frequency of projections of the modulated light pattern by the modulated light projector responsive to the ambient light condition.

3. The electronic device of claim 1, wherein the controller is to a control the intensity of projections of the modulated light pattern by the modulated light projector responsive to the ambient light condition.

4. The electronic device of claim 1, wherein the controller is to a control the duration of projections of the modulated light pattern by the modulated light projector responsive to the ambient light condition.

5. The electronic device of claim 1, further comprising:

a display device to display images captured by the first imaging camera.

6. The electronic device of claim 5, wherein the controller is to prevent display of images that were captured by the first imaging camera concurrent with capturing a reflection of the modulated light pattern.

18. The electronic device of claim 1, wherein the controller is to increase at least one of the frequency, the intensity, and the duration of the modulated light pattern projected responsive to the ambient light condition being less than a first threshold.

19. The electronic device of claim 1, wherein the controller is to decrease at least one of the frequency, the intensity, and the duration of the modulated light pattern projected responsive to the ambient light condition being greater than a second threshold.

7. A method comprising:

capturing images of an environment of an electronic device using a first imaging camera of the electronic device;

displaying images captured by the first imaging camera at a display of the electronic device;

projecting modulated light patterns using a modulated light projector of the electronic device, wherein at least one of a frequency, an intensity, and a duration of projections of the modulated light patterns is based on ambient lighting for the electronic device;

capturing reflections of the projected modulated light patterns using a second imaging camera;

determining spatial features in the environment based on at least one of the images captured by the first imaging camera and the reflections of the projected modulated light patterns captured by the second imaging camera;

increasing at least one of the frequency, the intensity, and the duration of the modulated light pattern projected responsive to the ambient lighting falling below a first threshold; and

decreasing at least one of the frequency, the intensity, and the duration of the modulated light pattern projected responsive to the ambient lighting rising above a second threshold.

8. The method of claim 7, wherein the frequency of projections of the modulated light pattern by the modulated light projector is based on the ambient lighting.

9. The method of claim 7, wherein the intensity of projections of the modulated light pattern by the modulated light projector is based on the ambient lighting.

10. The method of claim 7, wherein the duration of projections of the modulated light pattern by the modulated light projector is based on the ambient lighting.

11. The method of claim 7, further comprising:

preventing display at the electronic device of an image that was captured by the first imaging camera while a modulated light pattern was projected by the modulated light projector.

20. The method of claim 7, wherein the first imaging camera and the second imaging camera are the same imaging camera.

12. A method comprising:

projecting modulated light patterns using a modulated light projector of an electronic device;

capturing reflections of the projected modulated light patterns using an imaging camera;

controlling the modulated light projector to selectively modify at least one of a frequency, an intensity, and a duration of projections of the modulated light pattern responsive to a change in ambient lighting; and

wherein controlling the modulated light projector comprises at least one of;

increasing at least one of the frequency, the intensity, and the duration of the modulated light pattern projected responsive to the ambient lighting falling below a first threshold; and

decreasing at least one of the frequency, the intensity, and the duration of the modulated light pattern projected responsive to the ambient lighting rising above a second threshold.

13. The method of claim 12, wherein controlling the modulated light projector comprises controlling the modulated light projector to selectively modify the frequency of projections of the modulated light pattern by the modulated light projector responsive to the change in ambient lighting.

14. The method of claim 12, wherein controlling the modulated light projector comprises controlling the modulated light projector to selectively modify the intensity of projections of the modulated light pattern by the modulated light projector responsive to the change in ambient lighting.

15. The method of claim 12, wherein controlling the modulated light projector comprises controlling the modulated light projector to selectively modify the duration of projections of the modulated light pattern by the modulated light projector responsive to the change in ambient lighting.

16. The method of claim 12, further comprising:

capturing at least one image of an environment of the electronic device; and

determining at least one spatial feature based on the at least one image.

17. The method of claim 12, further comprising:

preventing display at the electronic device of an image that was captured by an imaging camera of the electronic device while a modulated light pattern was projected by the modulated light projector.