MULTIPLE EXPOSURE HIGH DYNAMIC RANGE IMAGE CAPTURE
First Claim
1. A method of mixing a series of two or more images of a scene, each series image composed of pixels containing digital bits to generate a composite image in which each pixel contains a number of digital bits, the number being greater than the number of digital bits contained in any series image pixel, comprising:
- capturing the series images at different exposure levels;
registering the counterpart pixels of each image of the image series one to another;
deriving a normalized image exposure level for each image in the series by using the exposure level of the series image that is least exposed as a reference level;
employing the normalized image exposure levels in an image blending process;
using the image blending process to blend together a first image of the series and a second image of the series, wherein the second image of the series is captured at a higher exposure level than the first image of the series, to generate an intermediate image, the intermediate image being a mixed output image when the series is composed of two images; and
repeating-the image blending process using the generated intermediate image in place of the first image of the series and a subsequent series image in place of the second image of the series when the series is composed of more than two images, each successively blended series image being exposed at a next higher exposure level than the previously blended series image, until all series images have been blended, the last generated intermediate image being the mixed output image.
5 Assignments
0 Petitions
Accused Products
Abstract
Techniques for creating a High Dynamic Range (HDR) image within a consumer grade digital camera from a series of images of a scene captured at different exposure levels, and displaying the HDR image on the camera'"'"'s built-in display, are provided. The approach employs mixing images of the series to incorporate both scene shadow and highlight details, and the removing of “ghost” image artifacts appearing in the mixed HDR image resulting from movement in the scene over the time the series images are captured. The low computational resource utilization of the present invention'"'"'s image mixing and ghost removal processing operations, along with the present invention'"'"'s ability to commence image mixing and ghost removal prior to the acquisition of all series images, can significantly reduce the time required to generate and display a tone mapped HDR image.
-
Citations
29 Claims
-
1. A method of mixing a series of two or more images of a scene, each series image composed of pixels containing digital bits to generate a composite image in which each pixel contains a number of digital bits, the number being greater than the number of digital bits contained in any series image pixel, comprising:
-
capturing the series images at different exposure levels; registering the counterpart pixels of each image of the image series one to another; deriving a normalized image exposure level for each image in the series by using the exposure level of the series image that is least exposed as a reference level; employing the normalized image exposure levels in an image blending process; using the image blending process to blend together a first image of the series and a second image of the series, wherein the second image of the series is captured at a higher exposure level than the first image of the series, to generate an intermediate image, the intermediate image being a mixed output image when the series is composed of two images; and repeating-the image blending process using the generated intermediate image in place of the first image of the series and a subsequent series image in place of the second image of the series when the series is composed of more than two images, each successively blended series image being exposed at a next higher exposure level than the previously blended series image, until all series images have been blended, the last generated intermediate image being the mixed output image. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. A method of removing location shifted replications of scene objects appearing in mixed image data generated by a digital image mixing process applied to a series of scene images acquired at different exposure levels and different times, such scene images having their counterpart pixels registered one to another, comprising:
-
selecting a reference image from one of the acquired scene images; determining a variance of the luma of each pixel of the mixed image data, as compared to its counterpart reference image data pixel; replacing each pixel of the mixed image data with a variance exceeding a first threshold level with its counterpart reference image data pixel, the resulting first processed mixed image data being the output of a first stage of processing; calculating the difference between the luma of each first processed mixed image data pixel and the luma of its counterpart reference image data pixel; generating a second threshold level based on the maximum value of the differences; and replacing each first processed mixed image data pixel exceeding the second threshold level with its counterpart reference image data pixel, the resulting second processed mixed image data being the output of a second stage of processing. - View Dependent Claims (9, 10, 11, 12)
-
-
13. An image capturing device that captures a series of two or more digital images of a scene at different exposure levels, each series image composed of pixels containing a number of digital bits, that includes:
-
an image registration processor that registers the counterpart pixels of each image of the image series one to another; an image mixer that combines multiple images of the series, such series images having their counterpart pixels registered one to another, to generate a single image whose pixels contain a number of digital bits, the number being greater than the number of digital bits contained in any series image pixel, wherein the mixer is comprised of; an image normalizer that normalizes the image exposure level for each image in the series by using the exposure level of the least exposed series image as a reference level; an image blender, that uses the normalized exposure level to blend together a first image of the series and a second image of the series, wherein the second image of the series is captured at a higher exposure level than the first image of the series, and generates a first intermediate image, the first intermediate image being the mixed output image from the blender when the series is composed of two images; an image blender to blend together the first intermediate image with a third image of the series, the third image of the series being exposed at a next higher exposure level, to generate a subsequent intermediate image, when the series is composed of more than two images; and an image blender to generate subsequent intermediate images by blending each derived intermediate image with a series image, each successively blended series image being exposed at a higher exposure level than the previously blended series image, until all series images have been blended, the last generated intermediate image being the mixed output image from the blender. - View Dependent Claims (14, 15, 16, 17, 18, 19)
-
-
20. An image capturing device that captures a series of two or more digital images of a scene at different exposure levels and different times, each series image composed of pixel data containing a number of digital bits, that includes:
-
an image registration processor that registers the counterpart pixels of each image of the image series one to another; an image mixer that combines multiple images of the series to generate as output mixed image data whose pixels contain a number of digital bits, the number being greater than the number of digital bits contained in any series image pixel and; a ghost remover that removes location shifted replications of scene objects appearing in the mixed image data output, wherein the ghost remover is comprised of; a reference image selection circuit that selects a reference image from one of the acquired scene images; a variance calculation circuit that calculates a variance of the luma of each pixel of the mixed image data, as compared to its counterpart reference image data pixel; a first pixel replacement circuit that replaces each mixed image data pixel with a variance that exceed a first threshold level, with its counterpart reference image data pixel, a first processed mixed image data being the output of the first pixel replacement circuit, the output being the result of a first stage of processing; a comparison circuit that calculates the difference between the luma of each first processed mixed image data pixel and the luma of its counterpart reference image data pixel, and generates a second threshold level based on the maximum value of the differences; and a second pixel replacement circuit that replaces each first processed mixed image data pixel exceeding the second threshold level with its counterpart reference image data pixel, a resulting second processed mixed image data being the output of a second stage of processing. - View Dependent Claims (21, 22, 23, 24)
-
-
25. A digital camera that captures a series of two or more digital images of a scene at different exposure levels, and at different times, and generates therefrom a tone mapped high dynamic range image, comprising:
-
an image registration processor that registers the counterpart pixels of each image of the image series one to another; an image mixer that combines multiple images of the series to generate a single image whose pixels contain a number of digital bits, the number being greater than the number of digital bits contained in any series image pixel, wherein the mixer is comprised of; an image normalizer that normalizes the image exposure level for each image in the series by using the exposure level of the least exposed image of the series as the reference level; an image blender, that uses the normalized exposure level to blend together a first image of the series and a second image of the series, wherein the first image of the series is captured at a lower exposure level than the second image of the series, and generates a first intermediate image, the first intermediate image being the mixed output image from the blender when the series is composed of two images; an image blender to blend together the first intermediate image with a third image of the series, the third image of the series being exposed at a next higher exposure level, to generate a subsequent intermediate image, when the series is composed of more than two images; and an image blender to generate subsequent intermediate images by blending each derived intermediate image with a series image, each successively blended series image being exposed at a higher exposure level than the previously blended series image, until all series images have been blended, the last generated intermediate image containing output mixed image data; a ghost remover that removes location shifted replications of scene objects appearing in the output mixed image data, wherein the ghost remover is comprised of; a reference image selection circuit that selects a reference image from one of the acquired series images; a variance calculation circuit that calculates a variance of the luma of every pixel of the mixed image data, as compared to pixels of the reference image; a first pixel replacement circuit that replaces pixels of the mixed image data with variances that exceed a first threshold value, with counterpart reference image pixels, first processed mixed image data pixels being the output of the first pixel replacement circuit, the output being the result of a first stage of processing; a comparison circuit that compares the luma of the first processed mixed image data pixels with the luma of counterpart reference image pixels, the comparison circuit calculating the differences between the luma of the first processed mixed image data pixels and the luma of the pixels of the reference image, and generates a second threshold value based on the maximum value of the differences; and a second pixel replacement circuit that replaces first processed mixed image data pixels exceeding the second threshold value with counterpart reference image pixels, the resulting second processed mixed image data being the output of a second stage of processing; and a tone mapping processor that maps the second process mixed image data pixels to image data pixels that contain the number of digital bits that can be presented on the digital camera'"'"'s built-in digital image display. - View Dependent Claims (26, 27, 28)
-
-
29. A method of mixing a series of two or more images of a scene, each series image composed of pixels containing digital bits to generate a composite image in which each pixel contains a number of digital bits, the number being greater than the number of digital bits contained in any series image pixel, comprising:
-
capturing the series images at different exposure levels; registering the counterpart pixels of each image of the image series one to another; deriving a normalized image exposure level for each image in the series by using the exposure level of the series image that is least exposed as a reference level; employing the normalized image exposure levels in an image blending process; using the image blending process to blend together a first image of the series and a second image of the series, wherein the second image of the series is captured at a higher exposure level than the first image of the series, to generate a first intermediate image, the first intermediate image being a mixed output image when the series is composed of two images; using the image blending process to blend together the first intermediate image with a third image of the series, the third image of the series being exposed at a next higher exposure level, to generate a second intermediate image, when the series is composed of more than two images; and using the image blending process to generate subsequent intermediate images by blending each generated intermediate image with a series image, each successively blended series image being exposed at a next higher exposure level than the previously blended series image, until all series images have been blended, the last generated intermediate image being the mixed output image.
-
Specification