Tone Mapping For Low-Light Video Frame Enhancement

• US 20150062387A1
• Filed: 11/06/2014
• Published: 03/05/2015
• Est. Priority Date: 03/05/2007
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Inventors: Vranceanu, Ruxandra
• Granted Patent: US 9,094,648 B2
• Time in Patent Office: Days
• US Class Current: 348/242
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10144 Varying exposure
• CPC Class Codes: G06T 2207/20008 Globally adaptive
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 5/92 based on global image properties
• CPC Class Codes: H04N 23/70 Circuitry for compensating brightness variation in the scene
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 9/646 for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters
• CPC Class Codes: H04N 9/68 for controlling the amplitude of colour signals, e.g. automatic chroma control circuits H04N9/71, H04N9/73 take precedence; camera processing pipelines for controlling the colour saturation of colour signals H04N23/86

Abstract

A technique is provided for generating sharp, well-exposed, color images from low-light images. A series of under-exposed images is acquired. A mean image is computed and a sum image is generated each based on the series of under-exposed images. Chrominance variables of pixels of the mean image are mapped to chrominance variables of pixels of the sum image. Chrominance values of pixels within the series of under-exposed images are replaced with chrominance values of the sum image. A set of sharp, well-exposed, color images is generated based on the series of under-exposed images with replaced chrominance values.

Claims

1. An image acquisition and processing device, comprising:

an image acquisition unit configured to;

acquire a plurality of sharp images of a scene; and

acquire a blurred image nominally of the same scene;

an image combining unit configured to;

generate a first combined image based, at least in part, on a first sharp image, from the plurality of sharp images, and color information of the blurred image; and

generate a second combined image based, at least in part, on a second sharp image, from the plurality of sharp images, and the first combined image;

wherein the first sharp image is different than the second sharp image; and

an image display unit configured to;

display the second combined image.

2. The image acquisition and processing device of claim 1,

wherein the first combined image is generated using also luminance information of the blurred image;

wherein the first combined image and a second combined image are color-enhanced and a luminance-enhanced versions of the first sharp image; and

wherein the second combined image appears less blurred than the blurred image.

3. The image acquisition and processing device of claim 1,

wherein pixels of the blurred image exhibit a greater motion blurring effect than pixels of the sharp image.

4. The image acquisition and processing device of claim 1, wherein the blurred image is acquired by digitally exposing pixels of a sensor array for an exposure time that is longer than respective exposure times during which each image of the plurality of sharp images is acquired.

5. The image acquisition and processing device of claim 1, wherein the blurred image nominally of the same scene is overexposed.

6. The image acquisition and processing device of claim 1,

wherein each image of the plurality of sharp images is represented in an RGB format;

and

wherein the blurred image is represented in the RGB format.

7. The image acquisition and processing device of claim 1,

wherein each image of the plurality of sharp images is represented in an YCC format;

and

wherein the blurred image is represented in the YCC format.

8. An image processing method comprising:

acquiring a plurality of sharp images of a scene;

acquiring a blurred image nominally of the same scene;

generating a first combined image based, at least in part, on a first sharp image, from the plurality of sharp images, and color information of the blurred image;

generating a second combined image based, at least in part, on a second sharp image, from the plurality of sharp images, and the first combined image;

wherein the first sharp image is different than the second sharp image; and

displaying the second combined image;

wherein the method is performed using one or more computing devices.

9. The image processing method of claim 8,

wherein the first combined image is generated using also luminance information of the blurred image;

wherein the first combined image and a second combined image are color-enhanced and a luminance-enhanced versions of the first sharp image; and

wherein the second combined image appears less blurred than the blurred image.

10. The image processing method of claim 8,

wherein pixels of the blurred image exhibit a greater motion blurring effect than pixels of the sharp image.

11. The image processing method of claim 8, wherein the blurred image is acquired by digitally exposing pixels of a sensor array for an exposure time that is longer than respective exposure times during which each image of the plurality of sharp images is acquired.

12. The image processing method of claim 8, wherein the blurred image nominally of the same scene is overexposed.

13. The image processing method of claim 8,

wherein each image of the plurality of sharp images is represented in an RGB format;

and

wherein the blurred image is represented in the RGB format.

14. The image processing method of claim 8,

wherein each image of the plurality of sharp images is represented in an YCC format;

and

wherein the blurred image is represented in the YCC format.

15. A non-transitory computer-readable storage medium storing one or more instructions which, when performed by one or more processors, cause the processors to perform:

acquiring a plurality of sharp images of a scene;

acquiring a blurred image nominally of the same scene;

generating a first combined image based, at least in part, on a first sharp image, from the plurality of sharp images, and color information of the blurred image;

generating a second combined image based, at least in part, on a second sharp image, from the plurality of sharp images, and the first combined image;

wherein the first sharp image is different than the second sharp image; and

displaying the second combined image.

16. The non-transitory computer-readable storage medium of claim 15,

wherein the first combined image is generated using also luminance information of the blurred image;

wherein the first combined image and a second combined image are color-enhanced and a luminance-enhanced versions of the first sharp image; and

wherein the second combined image appears less blurred than the blurred image.

17. The non-transitory computer-readable storage medium of claim 15,

wherein pixels of the blurred image exhibit a greater motion blurring effect than pixels of the sharp image.

18. The non-transitory computer-readable storage medium of claim 15, wherein the blurred image is acquired by digitally exposing pixels of a sensor array for an exposure time that is longer than respective exposure times during which each image of the plurality of sharp images is acquired.

19. The non-transitory computer-readable storage medium of claim 15, wherein the blurred image nominally of the same scene is overexposed.

20. The non-transitory computer-readable storage medium of claim 15,

wherein each image of the plurality of sharp images is represented in an YCC format;

and

wherein the blurred image is represented in the YCC format.