Local enhancement apparatus and method to generate high dynamic range images by blending brightness-preserved and brightness-adjusted blocks

• US 9,307,162 B2
• Filed: 05/21/2014
• Issued: 04/05/2016
• Est. Priority Date: 05/21/2014
• Status: Active Grant

• Current Assignee: Himax Imaging Limited (Himax Technologies Incorporated)
• Original Assignee: Himax Imaging Limited (Himax Technologies Incorporated)
• Inventors: Peng, Yuan-Chih
• Primary Examiner(s): Nazrul, Shahbaz
• Application Number:
  US14/284,215
• Publication Number:
  US 20150341537A1
• Time in Patent Office: 685 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/20208 High dynamic range [HDR] image processing
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: H04N 23/70 Circuitry for compensating brightness variation in the scene
• CPC Class Codes: H04N 23/71 Circuitry for evaluating the brightness variation
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
• CPC Class Codes: H04N 23/76 by influencing the image signals
• CPC Class Codes: H04N 25/53 Control of the integration time
• CPC Class Codes: H04N 25/58 involving two or more exposures

Abstract

An embodiment of a local enhancement apparatus is provided, in which a block divider divides an image into a plurality of blocks, and a first enhancement unit performs a brightness-preserved enhancement on each block of the plurality of blocks to generate a brightness-preserved block. A second enhancement unit performs a brightness-adjusted enhancement on the each block to generate a brightness-adjusted block. A scene dynamic range estimator generates a dynamic range of the image according to characteristics of the image. A blender blends the brightness-preserved blocks and the brightness-adjusted blocks to generate an enhanced image based on the dynamic range.

Claims

1. A local enhancement apparatus, comprising:

a block divider, for dividing an image into a plurality of blocks;

a first enhancement unit, for performing a brightness-preserved enhancement on each of the plurality of blocks respectively to generate a plurality of brightness-preserved blocks;

a second enhancement unit, for performing a brightness-adjusted enhancement on each of the plurality of blocks respectively to generate a plurality of brightness-adjusted blocks;

a scene dynamic range estimator, for generating a dynamic range of the image according to characteristics of the image; and

a blender, for blending the brightness-preserved blocks and the brightness-adjusted blocks to generate an enhanced image based on the dynamic range.

2. The local enhancement apparatus as claimed in claim 1, wherein the first enhancement unit performs histogram equalization on each of the plurality of blocks and preserves brightness of each of the plurality of blocks respectively to generate the brightness-preserved blocks and the second enhancement unit performs histogram equalization on each of the plurality of blocks and adjusts brightness of each of the plurality of blocks respectively to generate the brightness-adjusted blocks.

3. The local enhancement apparatus as claimed in claim 1, wherein the blender blends the brightness-preserved blocks and the brightness-adjusted blocks by a weighting based on the dynamic range.

4. The local enhancement apparatus as claimed in claim 1, wherein the brightness-preserved blocks are preferred by the blender when the dynamic range is far below a maximum dynamic range, and the brightness-adjusted blocks are preferred by the blender when the dynamic range is close to the maximum dynamic range.

5. A multi-exposure imaging system, comprising:

a long exposure image sensor, for capturing a long exposure image based on a long exposure from a scene;

a short exposure image sensor, for capturing a short exposure image based on a short exposure from the scene;

an exposure ratio calculator, for obtaining an exposure ratio of the long exposure to the short exposure; and

a local enhancement apparatus, for dividing a composite image into a plurality of blocks, performing a brightness-preserved enhancement and a brightness-adjusted enhancement on each of the plurality of blocks to generate a plurality of brightness-preserved blocks and a plurality of brightness-adjusted blocks respectively, and blending the brightness-preserved blocks and the brightness-adjusted blocks to generate an enhanced image based on the exposure ratio.

6. The multi-exposure imaging system as claimed in claim 5, further comprising:

an image composition unit, for generating the composite image according to the short exposure image and the long exposure image.

7. The multi-exposure imaging system as claimed in claim 5, wherein the local enhancement apparatus comprises a scene dynamic range estimator for obtaining a dynamic range of the composite image according to the exposure ratio, and the local enhancement apparatus blends the brightness-preserved blocks and the brightness-adjusted blocks to generate the enhanced image by a weighting according to the dynamic range.

8. The multi-exposure imaging system as claimed in claim 7, wherein the brightness-preserved blocks are preferred by the local enhancement apparatus when the dynamic range is far below a maximum dynamic range, and the brightness-adjusted blocks are preferred by the local enhancement apparatus when the dynamic range is close to the maximum dynamic range.

9. The multi-exposure imaging system as claimed in claim 7, wherein the dynamic range is substantially proportional to the exposure ratio.

10. The multi-exposure imaging system as claimed in claim 7, wherein the scene dynamic range estimator obtains the dynamic range by normalizing the exposure ratio.

11. The multi-exposure imaging system as claimed in claim 7, wherein the scene dynamic range estimator normalizes the exposure ratio to obtain the dynamic range by an equation of

12. The multi-exposure imaging system as claimed in claim 11, wherein the local enhancement apparatus further comprises:

a block divider for dividing the composite image into the plurality of blocks;

a first enhancement unit, for performing the brightness-preserved enhancement on each of the plurality of blocks respectively to generate the brightness-preserved blocks;

a second enhancement unit, for performing the brightness-adjusted enhancement on the each of the plurality of blocks respectively to generate the brightness-adjusted blocks; and

a blender, for blending the brightness-preserved blocks and the brightness-adjusted blocks to generate the enhanced image based on the dynamic range.

13. The multi-exposure imaging system as claimed in claim 12, wherein, when the dynamic range falls between 0 and 1, the first enhancement unit performs a brightness-preserved contrast enhancement with a first contrast enhancement function and the second enhancement unit performs a brightness-adjusted contrast enhancement with a second contrast enhancement function, and a final contrast enhancement function of the blender is expressed as CEFF=DR×

CEF2+(1−

DR)×

CEF1, wherein CEF1 is the first contrast enhancement function, CEF2 is the second contrast enhancement function, CEFF is the final contrast enhancement function and DR is the dynamic range.

14. The multi-exposure imaging system as claimed in claim 12, wherein the first enhancement unit performs histogram equalization on each of the plurality of blocks and preserves brightness of each of the plurality of blocks to generate the brightness-preserved blocks, and the second enhancement unit performs histogram equalization on each of the plurality of blocks and adjusts brightness of each of the plurality of blocks to generate the brightness-adjusted blocks.

19. The multi-exposure imaging system as claimed in claim 5, wherein only brightness-adjusted blocks are used to generate the enhanced image when the dynamic range is 1, and only the brightness-preserved blocks are used to generate the enhanced image when the dynamic range is 0.

15. A local enhancement method, comprising:

capturing a long exposure image and a short exposure image based on a long exposure and a short exposure respectively from a same scene;

combining the long exposure image and the short exposure image to obtain a composite image;

generating a dynamic range based on an exposure ratio of the long exposure to the short exposure;

performing a brightness-adjusted contrast enhancement on the composite image to generate a first contrast enhanced image;

performing a brightness-preserved contrast enhancement on the composite image to generate a second contrast enhanced image; and

blending the first contrast enhanced image and the second contrast enhanced image based on the dynamic range.

16. The local enhancement method as claimed in claim 15, wherein the dynamic range is substantially proportional to the exposure ratio.

17. The local enhancement method as claimed in claim 15, wherein the dynamic range is obtained by normalizing the exposure ratio.

18. The local enhancement method as claimed in claim 15, wherein the exposure ratio is normalized to obtain the dynamic range by an equation of