Using a second camera to adjust settings of first camera

• US 9,615,012 B2
• Filed: 09/30/2013
• Issued: 04/04/2017
• Est. Priority Date: 09/30/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Geiss, Ryan Burgess, Levoy, Marc, Lewkow, Roman
• Primary Examiner(s): Vaughn, Jr., William C
• Assistant Examiner(s): BELAI, NAOD W
• Application Number:
  US14/041,905
• Publication Number:
  US 20150092066A1
• Time in Patent Office: 1,282 Days
• Field of Search: 348180
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/70 Circuitry for compensating brightness variation in the scene
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors

Abstract

A device may operate a first image-capture system to capture first image data of a scene. While the first image-capture system is capturing the first image data, the device may operate a second image-capture system to determine an updated value for the first image setting, and send an instruction to the first image-capture system that indicates to use the updated value for the first image setting to continue to capture the first image data.

Claims

1. A method comprising:

operating, by a computing device, a first image-capture system to capture first image data of a scene, wherein the first image-capture system initially uses a first value for a first image setting to capture the first image data; and

while the first image-capture system is capturing the first image data;

operating, by the computing device, a second image-capture system to determine an updated value for the first image setting, wherein the first image-capture system and the second image-capture system are arranged on a given device and oriented in substantially the same direction; and

sending an instruction to the first image-capture system that indicates to use the updated value for the first image setting to continue to capture the first image data;

wherein the first image setting comprises an exposure time for the first image-capture system, and

wherein operating the second image-capture system to determine the updated value for the first image setting comprises;

(i) operating the second image-capture system to capture a second plurality of images of the scene, wherein each of the second plurality of images is captured using a different exposure time;

(ii) constructing a pixel-value histogram from the plurality of second images of the scene,

(iii) comparing the pixel value histogram to each of a plurality of reference pixel-value histograms to determine, for the pixel-value histogram and each of the reference pixel-value histograms, a respective similarity metric;

(iv) determining respective weights for each of the plurality of reference pixel-value histograms based on the respective similarity metrics; and

(v) determining an exposure time for use by the first camera system, based on the respective weights of the reference pixel-value histograms.

2. The method of claim 1, wherein determining the exposure time comprises determining a total exposure time (TET) for use by the first image-capture system to continue to capture the first image data.

3. The method of claim 1, wherein comparing the pixel-value histogram to each of the plurality of reference pixel-value histograms comprises:

determining, for the pixel-value histogram and each of the one or more reference pixel-value histograms, respective similarity metrics; and

determining respective weights based on inverses of the similarity metrics.

4. The method of claim 3, wherein the plurality of reference pixel-value histograms are associated with respective target average pixel values, and wherein the updated value for the exposure setting is determined based on a sum of the respective weights applied to the respective target average pixel values.

5. The method of claim 1, wherein operating the second image-capture system to determine an updated value for the first image setting further comprises:

(a) causing the second image-capture system to capture second image data using a test value for the first image setting;

(b) determining whether or not the second image data satisfies one or more image criteria;

(c) if the second image data satisfies the one or more criteria, then setting the first image setting to the test value; and

(d) otherwise, changing the test value for the first image setting, and then repeating (a) through (d) until it is determined at (c) that the second image data meets the one or more criteria.

6. The method of claim 1, wherein the first image data comprises video.

7. The method of claim 1, wherein the first image data comprises image data for display in a viewfinder.

8. The method of claim 1, wherein the first image setting comprises an exposure setting.

9. The method of claim 1, further comprising performing the method of claim 1 in order to update each of two or more image settings for the first-image capture system while the first image-capture system is capturing the first image data of the scene, wherein the two or more image settings comprise the first image setting and a second image setting.

10. The method of claim 1, wherein the second image-capture system is initially configured to capture still images, and wherein the method further comprises:

before starting to capture the first image data using the first image-capture system, reconfiguring the second image-capture system to determine updates for the first image setting on behalf of the first image-capture system.

11. The method of claim 9, wherein the computing device alternates between:

(a) operating the second image-capture system to update the first image setting and (b) operating the second image-capture system to update the second image setting.

12. The method of claim 9, wherein a third image-capture system is also arranged on the given device and oriented in substantially the same direction as both the first and second image-capture systems, and wherein the method further comprises:

while the first image-capture system is capturing the first image data;

operating, by the computing device, the third image-capture system to determine an updated value for the second image setting; and

sending an instruction to the first image-capture system that indicates to use the updated value for the second image setting to continue to capture the first image data.

13. The method of claim 9, wherein the two or more image settings comprise a focus setting and the exposure time.

14. The method of claim 9, wherein the two or more image settings comprise a near-field focus setting and a far-field focus setting.

15. The method of claim 9, wherein the first image setting comprises a focus setting, and wherein the second image setting comprises the exposure time.

16. A system comprising:

two or more image-capture systems that comprise at least a first and second image-capture systems, wherein the first and second image-capture systems are oriented in substantially the same direction;

a control system configured to;

initially operate the first image-capture system to capture first image data using a first value for a first image setting; and

while the first image-capture system is capturing the first image data;

operate the second image-capture system to determine an updated value for the first image setting; and

cause the first image-capture system to use the updated image value for the first image setting to continue to capture the first image data

wherein the first image setting comprises an exposure time for the first image-capture system, and

wherein, to operate of the second image-capture system to determine the updated value for the first image setting, the control system is configured to;

(i) operate the second image-capture system to capture a second plurality of images of the scene, wherein each of the second plurality of images is captured using a different exposure time;

(ii) construct a pixel-value histogram from the plurality of second images of the scene,

(iii) compare the pixel value histogram to each of a plurality of reference pixel-value histograms to determine, for the pixel-value histogram and each of the reference pixel-value histograms, a respective similarity metric;

(iv) determine respective weights for each of the plurality of reference pixel-value histograms based on the respective similarity metrics; and

(v) determine an exposure time for use by the first camera system, based on the respective weights of the reference pixel-value histograms.

17. The system of claim 16, wherein the determined value for the exposure time comprises a total exposure time (TET) for use by the first image-capture system to continue to capture the first image data.

18. The system of claim 16, wherein, to operate of the second image-capture system to determine the updated value for the first image setting, the control system is further configured to:

(a) operate the second image-capture system to capture second image data of the scene using a test value for the image setting;

(b) determine whether or not the second image data satisfies one or more image criteria;

(c) if the second image data satisfies the one or more criteria, then set the image setting to the test value; and

(d) otherwise, change the test value for the image setting and repeat (a) through (d) until it is determined that the second image data meets the one or more criteria.

19. The system of claim 16, wherein the first image data comprises video.

20. The system of claim 16, wherein the system is a mobile device, and wherein the first and the second image-capture system are arranged at a corner of a first surface of the mobile device.

21. The system of claim 16, wherein the system is a mobile device, and wherein the mobile device includes four image-capture systems that are oriented in the same direction, wherein two of the four image-capture systems are arranged at a first corner of a first surface of the mobile device, and wherein a third and a fourth of the four image-capture systems are arranged at a second and third corner of the first surface, respectively.

22. The system of claim 16, wherein the system is a mobile device, and wherein the mobile device includes four image-capture systems that are oriented in the same direction, wherein one of the four image-capture systems is arranged at each of four corners of a first surface of the mobile device.

23. A non-transitory computer readable medium having stored therein instructions executable by a computing device to cause the computing device to perform functions comprising:

operating a first image-capture system to capture first image data of a scene, wherein the first image-capture system initially uses a first value for a first image setting to capture the first image data; and

while the first image-capture system is capturing the first image data;

operating a second image-capture system to determine an updated value for the first image setting, wherein the first image-capture system and the second image-capture system are arranged on a given device and oriented in substantially the same direction; and

sending an instruction to the first image-capture system that indicates to use the updated value for the first image setting to continue to capture the first image data;

wherein the first image setting comprises an exposure time for the first image-capture system, and

wherein operating the second image-capture system to determine an updated value for the first image setting comprises;

(i) operating the second image-capture system to capture a second plurality of images of the scene, wherein each of the second plurality of images is captured using a different exposure time;

(ii) constructing a pixel-value histogram from the plurality of second images of the scene,

(iii) comparing the pixel value histogram to each of a plurality of reference pixel-value histograms to determine, for the pixel-value histogram and each of the reference pixel-value histograms, a respective similarity metric;

(iv) determining respective weights for each of the plurality of reference pixel-value histograms based on the respective similarity metrics; and

(v) determining an exposure time for use by the first camera system, based on the respective weights of the reference pixel-value histograms.

24. The non-transitory computer readable medium of claim 23, wherein determining a value for the exposure time comprises determining a total exposure time (TET) for use by the first image-capture system to continue to capture the first image data.