Color correcting virtual long exposures with true long exposures

• US 9,131,201 B1
• Filed: 05/24/2013
• Issued: 09/08/2015
• Est. Priority Date: 05/24/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Geiss, Ryan
• Primary Examiner(s): Berhan, Ahmed A
• Application Number:
  US13/902,267
• Time in Patent Office: 837 Days
• Field of Search: 348/362, 348/363, 348/262, 348/263, 348/222.1, 348/221.1, 348/272, 348/225.1, 348/229.1, 348/220.1
• US Class Current: 1/1
• CPC Class Codes: H04N 23/70 Circuitry for compensating brightness variation in the scene
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
• CPC Class Codes: H04N 25/589 with different integration times, e.g. short and long exposures

Abstract

A gain may be applied to a short-exposure image to form a virtual long-exposure image. The short-exposure image may have been captured using a short total exposure time (TET), the virtual long-exposure image may have a virtual long TET, and the virtual long TET may be greater than the short TET. A true long-exposure image may be captured using a true long TET. The true long TET may be substantially equal to the virtual long TET. Color values of at least some pixels of the true long-exposure image may be copied to corresponding pixels of the virtual long-exposure image.

Claims

1. A method comprising:

applying, by a computing device, gain to a short-exposure image to form a virtual long-exposure image, wherein the short-exposure image was captured using a short total exposure time (TET), the virtual long-exposure image has a virtual long TET, and the virtual long TET is greater than the short TET;

capturing a true long-exposure image using a true long TET, wherein the true long TET is substantially equal to the virtual long TET;

copying color values of at least some pixels of the true long-exposure image to corresponding pixels of the virtual long-exposure image; and

combining the short-exposure image and the virtual long-exposure image to form an output image, wherein more of the output image is properly-exposed than the short-exposure image, and wherein more of the output image is properly-exposed than the virtual long-exposure image.

2. The method of claim 1, wherein the true long TET is equal to the virtual long TET.

3. The method of claim 1, wherein the true long TET has an actual exposure time greater than that of the virtual long TET, and a gain that is less than that of the virtual long TET.

4. The method of claim 1, further comprising:

capturing a plurality of preliminary short-exposure images using the short TET; and

combining the plurality of preliminary short-exposure images to form the short-exposure image.

5. The method of claim 4, wherein the plurality of preliminary short-exposure images and the true long-exposure image are captured within a time span of 1 second or less, but more than 0 seconds.

6. The method of claim 1, wherein copying the color values of at least some pixels of the true long-exposure image to the corresponding pixels of the virtual long-exposure image comprises copying low-frequency characteristics of the color values to the corresponding pixels of the virtual long-exposure image.

7. The method of claim 1, wherein the gain applies a multiple of between 1 and 5 to at least some pixel values of the short-exposure image.

8. An article of manufacture including a non-transitory computer-readable medium, having stored thereon program instructions that, upon execution by a computing device, cause the computing device to perform operations comprising:

applying gain to a short-exposure image to form a virtual long-exposure image, wherein the short exposure image was captured using a short total exposure time (TET), the virtual long-exposure image has a virtual long TET, and the virtual long TET is greater than the short TET;

capturing a true long-exposure image using a true long TET, wherein the true long TET is substantially equal to the virtual long TET;

copying color values of at least some pixels of the true long-exposure image to corresponding pixels of the virtual long-exposure image; and

combining the short-exposure image and the virtual long-exposure image to form an output image, wherein more of the output image is properly-exposed than the short-exposure image, and wherein more of the output image is properly-exposed than the virtual long-exposure image.

9. The article of manufacture of claim 8, wherein the true long TET is equal to the virtual long TET.

10. The article of manufacture of claim 8, wherein the true long TET has an actual exposure time greater than that of the virtual long TET, and a gain that is less than that of the virtual long TET.

11. The article of manufacture of claim 8, the operations further comprising:

capturing a plurality of preliminary short-exposure images using the short TET; and

combining the plurality of preliminary short-exposure images to form the short-exposure image.

12. The article of manufacture of claim 11, wherein the plurality of preliminary short-exposure images and the true long-exposure image are captured within a time span of 1 second or less, but more than 0 seconds.

13. The article of manufacture of claim 8, wherein copying the color values of at least some pixels of the true long-exposure image to the corresponding pixels of the virtual long-exposure image comprises copying low-frequency characteristics of the color values to the corresponding pixels of the virtual long-exposure image.

14. The article of manufacture of claim 8, wherein the gain applies a multiple of between 1 and 5 to at least some pixel values of the short-exposure image.

15. A computing device comprising:

at least one processor;

data storage; and

program instructions, stored in the data storage, that upon execution by the at least one processor cause the computing device to perform operations including;

applying gain to a short-exposure image to form a virtual long-exposure image, wherein the short exposure image was captured using a short total exposure time (TET), the virtual long-exposure image has a virtual long TET, and the virtual long TET is greater than the short TET;

capturing a true long-exposure image using a true long TET, wherein the true long TET is substantially equal to the virtual long TET;

copying color values of at least some pixels of the true long-exposure image to corresponding pixels of the virtual long-exposure image; and

combining the short-exposure image and the virtual long-exposure image to form an output image, wherein more of the output image is properly-exposed than the short-exposure image, and wherein more of the output image is properly-exposed than the virtual long-exposure image.

16. The computing device of claim 15, wherein the true long TET is equal to the virtual long TET.

17. The computing device of claim 15, wherein the true long TET has an actual exposure time greater than that of the virtual long TET, and a gain that is less than that of the virtual long TET.

18. The computing device of claim 15, wherein copying the color values of at least some pixels of the true long-exposure image to the corresponding pixels of the virtual long-exposure image comprises copying low-frequency characteristics of the color values to the corresponding pixels of the virtual long-exposure image.

19. The computing device of claim 15, the operations further including:

capturing a plurality of preliminary short-exposure images using the short TET; and

combining the plurality of preliminary short-exposure images to form the short-exposure image.

20. The computing device of claim 19, wherein the plurality of preliminary short-exposure images and the true long-exposure image are captured within a time span of 1 second or less, but more than 0 seconds.