Simulating high dynamic range imaging with virtual long-exposure images

• US 9,077,913 B2
• Filed: 05/24/2013
• Issued: 07/07/2015
• Est. Priority Date: 05/24/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Hasinhoff, Samuel William, Geiss, Ryan
• Primary Examiner(s): Moe, Aung S
• Assistant Examiner(s): COWAN, EUEL K
• Application Number:
  US13/902,253
• Publication Number:
  US 20140347521A1
• Time in Patent Office: 774 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
• CPC Class Codes: H04N 5/265 Mixing

Abstract

A total exposure time (TET) may be selected. A plurality of images of a scene may be captured using respective TETs that are based on the selected TET. At least two of the images in the plurality of images may be combined to form a merged short-exposure image. A digital gain may be applied to the merged short-exposure image to form a virtual long-exposure image. The merged short-exposure image and the virtual long-exposure image may be combined to form an output image. More of the output image may be properly-exposed than either of the merged short-exposure image or the virtual long-exposure image.

Claims

1. A method comprising:

selecting a total exposure time (TET);

capturing, by an image capture device, a plurality of images of a scene, wherein the images in the plurality of images are captured using respective TETs that are based on the selected TET;

combining at least two of the images in the plurality of images to form a merged short-exposure image;

applying a digital gain to at least some pixels of the merged short-exposure image to form a virtual long-exposure image; and

combining the merged short-exposure image and the virtual long-exposure image to form an output image, wherein more of the output image is properly-exposed than either of the merged short-exposure image or the virtual long-exposure image.

2. The method of claim 1, further comprising:

determining that a dynamic range of the scene is between a lower dynamic range threshold and an upper dynamic range threshold, wherein applying the digital gain to at least some pixels of the merged short-exposure image and combining the merged short-exposure image and the virtual long-exposure image are in response to determining that the dynamic range of the scene is between the lower dynamic range threshold and the upper dynamic range threshold.

3. The method of claim 2, wherein the virtual long-exposure image is associated with a virtual long TET, and wherein the dynamic range of the scene corresponds to a ratio of the virtual long TET to the selected TET.

4. The method of claim 3, wherein the lower dynamic range threshold is between 1.25 and 2, and the upper dynamic range threshold is between 3.5 and 7.

5. The method of claim 1, wherein the virtual long-exposure image is one of a set of virtual long-exposure images formed by applying different digital gains to the merged short-exposure image, and wherein the output image is formed by combining the merged short-exposure image and the set of virtual long-exposure images.

6. The method of claim 1, wherein between 2 and 30 images from the plurality of images are combined to form the merged short-exposure image.

7. The method of claim 6, wherein between 3 and 10 images from the plurality of images are combined to form the merged short-exposure image.

8. The method of claim 1, wherein the plurality of images is captured within a time span of 1 second or less.

9. The method of claim 1, wherein the digital gain applies a multiple of between 1 and 5 to at least some pixel values of the merged short-exposure image.

10. The method of claim 1, wherein a dynamic range of the merged short-exposure image represents a range of pixel brightnesses in the merged short-exposure image.

11. An article of manufacture including a non-transitory computer-readable medium, having stored thereon program instructions that, upon execution by a computing device, cause the computing device to perform operations comprising:

selecting a total exposure time (TET);

capturing a plurality of images of a scene, wherein the images in the plurality of images are captured using respective TETs that are based on the selected TET;

combining at least two of the images in the plurality of images to form a merged short-exposure image;

applying a digital gain to the merged short-exposure image to form a virtual long-exposure image; and

combining the merged short-exposure image and the virtual long-exposure image to form an output image, wherein more of the output image is properly-exposed than either of the merged short-exposure image or the virtual long-exposure image.

12. The article of manufacture of claim 11, wherein the operations further comprise:

determining that a dynamic range of the scene is between a lower dynamic range threshold and an upper dynamic range threshold, wherein applying the digital gain to the merged short-exposure image and combining the merged short-exposure image and the virtual long-exposure image are in response to determining that the dynamic range of the scene is between the lower dynamic range threshold and the upper dynamic range threshold.

13. The article of manufacture of claim 12, wherein the virtual long-exposure image is associated with a virtual long TET, and wherein the dynamic range of the scene corresponds to a ratio of the virtual long TET to the selected TET.

14. The article of manufacture of claim 13, wherein the lower dynamic range threshold is between 1.25 and 2, and the upper dynamic range threshold is between 3.5 and 7.

15. The article of manufacture of claim 11, wherein the selected TET is selected based on one or more metering images captured by the computing device.

16. A computing device comprising:

at least one processor;

data storage; and

program instructions, stored in the data storage, that upon execution by the at least one processor cause the computing device to perform operations including;

selecting a total exposure time (TET);

capturing a plurality of images of a scene, wherein the images in the plurality of images are captured using respective TETs that are based on the selected TET;

combining at least two of the images in the plurality of images to form a merged short-exposure image;

applying a digital gain to the merged short-exposure image to form a virtual long-exposure image; and

combining the merged short-exposure image and the virtual long-exposure image to form an output image, wherein more of the output image is properly-exposed than either of the merged short-exposure image or the virtual long-exposure image.

17. The computing device of claim 16, wherein the operations further comprise:

determining that a dynamic range of the scene is between a lower dynamic range threshold and an upper dynamic range threshold, wherein applying the digital gain to the merged short-exposure image and combining the merged short-exposure image and the virtual long-exposure image are in response to determining that the dynamic range of the scene is between the lower dynamic range threshold and the upper dynamic range threshold.

18. The computing device of claim 17, wherein the virtual long-exposure image is associated with a virtual long TET, and wherein the dynamic range of the scene corresponds to a ratio of the virtual long TET to the selected TET.

19. The computing device of claim 17, wherein the lower dynamic range threshold is between 1.25 and 2, and the upper dynamic range threshold is between 3.5 and 7.

20. The computing device of claim 16, wherein the selected TET is selected based on one or more metering images captured by the computing device.