Using a Second Camera to Adjust Settings of First Camera

• US 20150092066A1
• Filed: 09/30/2013
• Published: 04/02/2015
• Est. Priority Date: 09/30/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Levoy, Marc, Geiss, Ryan Burgess, Lewkow, Roman
• Granted Patent: US 9,615,012 B2
• Time in Patent Office: Days
• US Class Current: 348/180
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/70 Circuitry for compensating brightness variation in the scene
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors

Abstract

A device may operate a first image-capture system to capture first image data of a scene. While the first image-capture system is capturing the first image data, the device may operate a second image-capture system to determine an updated value for the first image setting, and send an instruction to the first image-capture system that indicates to use the updated value for the first image setting to continue to capture the first image data.

Claims

1. A method comprising:

operating, by a computing device, a first image-capture system to capture first image data of a scene, wherein the first image-capture system initially uses a first value for a first image setting to capture the first image data; and

while the first image-capture system is capturing the first image data;

operating, by the computing device, a second image-capture system to determine an updated value for the first image setting, wherein the first image-capture system and the second image-capture system are arranged on a given device and oriented in substantially the same direction; and

sending an instruction to the first image-capture system that indicates to use the updated value for the first image setting to continue to capture the first image data.

2. The method of claim 1, wherein the first image setting comprises an exposure setting for the first image-capture system, and wherein operating the second image-capture system to determine an updated value for the first image setting comprises:

operating the second image-capture system to capture a plurality of second images of a scene, wherein each image of the plurality of images is captured using a different value for the exposure setting;

downsampling the plurality of second images;

constructing a pixel-value histogram based on pixel values of the downsampled images;

comparing the pixel-value histogram to one or more reference pixel-value histograms; and

determining an updated value for the exposure setting based on comparing the pixel-value histogram to the one or more reference pixel-value histograms.

3. The method of claim 2, wherein determining a value for the exposure setting comprises determining a total exposure time (TET) for use by the first image-capture system to continue to capture the first image data.

4. The method of claim 2, wherein comparing the pixel-value histogram to the one or more reference pixel-value histograms comprises:

determining, for the pixel-value histogram and each of the one or more reference pixel-value histograms, respective similarity metrics; and

determining respective weights based on inverses of the similarity metrics.

5. The method of claim 4, wherein the one or more reference pixel-value histograms are associated with respective target average pixel values, and wherein the updated value for the exposure setting is determined based on a sum of the respective weights applied to the respective target average pixel values.

6. The method of claim 1, wherein operating the second image-capture system to determine an updated value for the first image setting comprises:

(a) causing the second image-capture system to capture second image data using a test value for the first image setting;

(b) determining whether or not the second image data satisfies one or more image criteria;

(c) if the second image data satisfies the one or more criteria, then setting the first image setting to the test value; and

(d) otherwise, changing the test value for the first image setting, and then repeating (a) through (d) until it is determined at (c) that the second image data meets the one or more criteria.

7. The method of claim 1, wherein the first image data comprises video.

8. The method of claim 1, wherein the first image data comprises image data for display in a viewfinder.

9. The method of claim 1, wherein the first image setting comprises a focus setting.

10. The method of claim 1, wherein the first image setting comprises an exposure setting.

11. The method of claim 1, wherein the first image setting comprises a white-balance setting.

12. The method of claim 1, further comprising performing the method of claim 1 in order to update each of two or more image settings for the first-image capture system while the first image-capture system is capturing the first image data of the scene, wherein the two or more image settings comprise the first image setting and a second image setting.

13. The method of claim 12, wherein the computing device alternates between:

(a) operating the second image-capture system to update the first image setting and (b) operating the second image-capture system to update the second image setting.

14. The method of claim 12, wherein a third image-capture system is also arranged on the given device and oriented in substantially the same direction as both the first and second image-capture systems, and wherein the method further comprises:

while the first image-capture system is capturing the first image data;

operating, by the computing device, the third image-capture system to determine an updated value for the second image setting; and

sending an instruction to the first image-capture system that indicates to use the updated value for the second image setting to continue to capture the first image data.

15. The method of claim 12, wherein the two or more image settings comprise a focus setting and an exposure setting.

16. The method of claim 12, wherein the two or more image settings comprise a near-field focus setting and a far-field focus setting.

17. The method of claim 12, wherein the first image setting comprises a focus setting, and wherein the second image setting comprises an exposure setting.

18. The method of claim 1, wherein the second image-capture system is initially configured to capture still images, and wherein the method further comprises:

before starting to capture the first image data using the first image-capture system, reconfiguring the second image-capture system to determine updates for the first image setting on behalf of the first image-capture system.

19. A system comprising:

two or more image-capture systems that comprise at least a first and second image-capture systems, wherein the first and second image-capture systems are oriented in substantially the same direction;

a control system configured to;

initially operate the first image-capture system to capture first image data using a first value for a first image setting; and

while the first image-capture system is capturing the first image data;

operate the second image-capture system to determine an updated value for the first image setting; and

cause the first image-capture system to use the updated image value for the first image setting to continue to capture the first image data.

20. The system of claim 19, wherein, to operate of the second image-capture system to determine the updated value for the first image setting, the control system is configured to:

operate the second image-capture system to capture a plurality of second images of a scene, wherein each image of the plurality of images is captured using a different value for the exposure setting;

downsample the plurality of second images;

construct a pixel-value histogram based on pixel values of the downsampled images;

compare the pixel-value histogram to one or more reference pixel-value histograms; and

determine an updated value for the exposure setting based on comparing the pixel-value histogram to the one or more reference pixel-value histograms.

21. The system of claim 20, wherein the determined value for the exposure setting comprises a total exposure time (TET) for use by the first image-capture system to continue to capture the first image data.

22. The system of claim 19, wherein, to operate of the second image-capture system to determine the updated value for the first image setting, the control system is configured to:

(a) operate the second image-capture system to capture second image data of the scene using a test value for the image setting;

(b) determine whether or not the second image data satisfies one or more image criteria;

(c) if the second image data satisfies the one or more criteria, then set the image setting to the test value; and

(d) otherwise, change the test value for the image setting and repeat (a) through (d) until it is determined that the second image data meets the one or more criteria.

23. The system of claim 19, wherein the first image data comprises video.

24. The system of claim 19, wherein the image setting comprises one of:

(a) a focus setting, (b) an exposure setting, and (c) a white-balance setting.

25. The system of claim 19, wherein the system is a mobile device, and wherein the first and the second image-capture system are arranged at a corner of a first surface of the mobile device.

26. The system of claim 19, wherein the system is a mobile device, and wherein the mobile device includes four image-capture systems that are oriented in the same direction, wherein two of the four image-capture systems are arranged at a first corner of a first surface of the mobile device, and wherein a third and a fourth of the four image-capture systems are arranged at a second and third corner of the first surface, respectively.

27. The system of claim 19, wherein the system is a mobile device, and wherein the mobile device includes four image-capture systems that are oriented in the same direction, wherein one of the four image-capture systems is arranged at each of four corners of a first surface of the mobile device.

28. A non-transitory computer readable medium having stored therein instructions executable by a computing device to cause the computing device to perform functions comprising:

operating a first image-capture system to capture first image data of a scene, wherein the first image-capture system initially uses a first value for a first image setting to capture the first image data; and

while the first image-capture system is capturing the first image data;

operating a second image-capture system to determine an updated value for the first image setting, wherein the first image-capture system and the second image-capture system are arranged on a given device and oriented in substantially the same direction; and

sending an instruction to the first image-capture system that indicates to use the updated value for the first image setting to continue to capture the first image data.

29. The non-transitory computer readable medium of claim 28, wherein the first image setting comprises an exposure setting for the first image-capture system, and wherein operating the second image-capture system to determine an updated value for the first image setting comprises:

operating the second image-capture system to capture a plurality of second images of a scene, wherein each image of the plurality of images is captured using a different value for the exposure setting;

downsampling the plurality of second images;

constructing a pixel-value histogram based on pixel values of the downsampled images;

comparing the pixel-value histogram to one or more reference pixel-value histograms; and

determining an updated value for the exposure setting based on comparing the pixel-value histogram to the one or more reference pixel-value histograms.

30. The non-transitory computer readable medium of claim 29, wherein determining a value for the exposure setting comprises determining a total exposure time (TET) for use by the first image-capture system to continue to capture the first image data.