Method and system for real-time noise removal and image enhancement of high-dynamic range images

• US 10,672,112 B2
• Filed: 03/02/2016
• Issued: 06/02/2020
• Est. Priority Date: 03/05/2015
• Status: Active Grant

• Current Assignee: IEE International Electronics & Engineering SA (Aerospace Hi-Tech Holding Group Company Limited)
• Original Assignee: IEE International Electronics & Engineering SA (Aerospace Hi-Tech Holding Group Company Limited)
• Inventors: Garcia Becerro, Frederic, Schockaert, Cedric, Mirbach, Bruno
• Primary Examiner(s): Patel, Pinalben
• Application Number:
  US15/555,187
• Publication Number:
  US 20180053289A1
• Time in Patent Office: 1,553 Days
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation
• CPC Class Codes: G06T 2207/20208 High dynamic range [HDR] image processing
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/92 based on global image properties

Abstract

A method and system for real-time noise removal and image enhancement of high-dynamic range (HDR) images. The method includes receiving an HDR input image I and operating processing circuitry for (i) applying a first edge-preserving filter (e.g. guided filter) to the input image I, thereby generating a first image component B₁ and a first set of linear coefficients ā_i,1; (ii) applying a second edge-preserving filter (e.g. guided filter) to the input image I, thereby generating a second image component B₂ and a second set of linear coefficients ā_i,2; (iii) generating a plausibility mask P from a combination of the first set of linear coefficients ā_i,1 and the second set of linear coefficients ā_i,2, the plausibility mask P indicating spatial detail within the input image I; and (iv) generating an output image O based on first image component B₁, the second image component B₂ and the plausibility mask P.

Claims

1. A method for real-time noise removal and image enhancement of high-dynamic range HDR images, the method comprising:

receiving an input image I, the input image I comprising a HDR image;

operating processing circuitry for

applying a first edge-preserving filter to the input image I, thereby generating a first image component B₁ and a first set of linear coefficients ā

_i,1;

said method is characterized by;

applying a second edge-preserving filter to the input image I, thereby generating a second image component B₂ and a second set of linear coefficients ā

_i,2, wherein the first edge-preserving filter and the second edge-preserving filter are distinct filters having different filtering characteristics such that the first image component B₁ is different than the second image component B₂, and the first set of linear coefficients ā

_i,1is different than the second set of linear coefficients ā

_i,2;

generating a plausibility mask P from a combination of the first set of linear coefficients ā

_i,1and the second set of linear coefficients ā

_i,2, wherein the plausibility mask is derived as;

P(i,j)=P′

(i,j)−

P′

_min

where P′

=ā

_i,1·

ā

_i,2and P′

_minand P′

_max the minimum and maximum weight values of P′

, respectively; and

generating an output image O based on first image component B₁, the second image component B₂ and the plausibility mask P.

2. The method of claim 1, wherein the first edge-preserving filter comprises a detail preserving filter and the first image component B₁ comprises the input image I in which image detail is preserved.

3. The method of claim 1, wherein the second edge-preserving filter comprises a noise removal filter and the second image component B₂ comprises the input image I from which noise has been removed or reduced.

4. The method of claim 1, wherein the first edge-preserving filter and/or the second edge-preserving filter comprises a guided filter.

5. The method of claim 1, further comprising cropping P′

before stretching its values according to

6. The method of claim 1, further comprising cropping P′

according to

7. The method of claim 1, wherein generating an output image O comprises generating an output image O based on the summation of the second image component B₂ and a detail image component D′

derived from the first image component B₁ and the plausibility mask P.

8. The method of claim 7, wherein the detail image component D′

=λ

P·

D, where D=B₁ −

B₂, and λ

is a gain factor for adjusting the contrast of details in D.

9. The method of claim 7, wherein generating an output image O further comprises applying an intermediate image O′

derived by

O′

=B₂+D′

.

10. The method of claim 1, wherein generating an output image O further comprises applying a mapping of an intermediate image O′

to output image O while preserving and enhancing the contrast of the image details in a detail image component D′

.

11. The method of claim 10, wherein the mapping comprises

generating a histogram of the grayscale value of each O′

(i,j) within the intermediate image O′

.

12. The method of claim 10, wherein the mapping comprises

classifying the grayscale value of each O′

(i, j) within the intermediate image O′

as left O′

_L(i,j) or right O′

_R(i, j) depending on its location within the histogram of O′

, such that

O′

_L(i,j)=O′

(i,j)·

α

ifO′

(i,j)·

α

<

λ

_med

O′

_R(i,j)=O′

(i,j)·

α

ifO′

(i,j)·

α

≥

λ

_med

where λ

_med corresponds to the location in the cumulative histogram of O′

·

α

at 50% and a is a contrast level.

13. The method of claim 10, wherein the mapping comprises determining output image O from

14. The method of claim 13, wherein N=8 for 8 bit data representation within the range [0, 255].

15. The method of claim 9, wherein the intermediate image O′

is a 12-bit or 14-bit image.

16. The method of claim 1, wherein for the first image component B₁ and the second image component B₂, each is derived using a linear transform of I in a window w_k centered at pixel k, such that

B_i=ā

_lI_i+b_v∀

iϵ

w_k,

where

17. The method of claim 16, wherein the size of w_k is 3×

3, ε

₁ =25 and/or ε

₂=2500.

18. A programmable image processing system when suitably programmed for carrying out the method of claim 1, the system comprising circuitry for receiving or storing an input image I and processing circuitry for performing the method.

20. A non-transitory computer-readable medium having recorded or stored thereon data defining or transformable into instructions for execution by processing circuitry and corresponding to at least the steps of claim 1.

21. A server computer incorporating a communications device and a memory device and being adapted for transmission on demand or otherwise of data defining or transformable into instructions for execution by processing circuitry and corresponding to at least the steps of claim 1.

19. A system for real-time noise removal and image enhancement of high-dynamic range HDR images, the system comprising:

circuitry for receiving or storing an input image I, the input image I comprising a HDR image;

processing circuitry, coupled to the circuitry for receiving or storing an input image I, the processing circuitry being operable for

applying a first edge-preserving filter to the input image I, thereby generating a first image component B₁ and a first set of linear coefficients ā

_i,1;

applying a second edge-preserving filter to the input image I, thereby generating a second image component B₂ and a second set of linear coefficients ā

_i,2, wherein the first edge-preserving filter and the second edge-preserving filter are distinct filters having different filtering characteristics such that the first image component B₁ is different than the second image component B₂, and the first set of linear coefficients ā

_i,1is different than the second set of linear coefficients ā

_i,2;

generating a plausibility mask P from a combination of the first set of linear coefficients ā

_i,1and the second set of linear coefficients ā

_i,2, wherein the plausibility mask is derived as;

P(i,j)=P′

(i,j)−

P′

_min

where P′

=ā

_i,1·

ā

_i,2and P′

_minand P′

_max the minimum and maximum weight values of P′

, respectively; and

generating an output image O based on first image component B₁, the second image component B₂ and the plausibility mask P.