Tone mapping method

• US 10,229,484 B2
• Filed: 11/30/2016
• Issued: 03/12/2019
• Est. Priority Date: 11/30/2016
• Status: Active Grant

• Current Assignee: STMicroelectronics Grenoble 2 SAS (STMicroelectronics NV)
• Original Assignee: STMicroelectronics Grenoble 2 SAS (STMicroelectronics NV)
• Inventors: Roffet, Gregory, Thivin, Mathieu
• Primary Examiner(s): Krasnic, Bernard
• Application Number:
  US15/365,086
• Publication Number:
  US 20180150946A1
• Time in Patent Office: 832 Days
• Field of Search: None
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20072 Graph-based image processing
• CPC Class Codes: G06T 2207/20208 High dynamic range [HDR] image processing
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 5/90 Dynamic range modification of images or parts thereof
• CPC Class Codes: G06T 5/92 based on global image properties
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: H04N 23/10 for generating image signals from different wavelengths
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/76 by influencing the image signals
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 9/68 for controlling the amplitude of colour signals, e.g. automatic chroma control circuits H04N9/71, H04N9/73 take precedence; camera processing pipelines for controlling the colour saturation of colour signals H04N23/86

Abstract

Tone mapping is applied to pixels of a digital image. A luminance value of a pixel is determined based on whether one or more pixel intensity values of a pixel in a color space are within a pixel saturation range. A pixel gain is determined based on the determined luminance value of the pixel, and the determined pixel gain is applied to the pixel. The luminance value may also or instead be determined based on whether one or more of the pixel intensity values is within a pixel black-out range. A weight may be employed to determine the luminance value.

Claims

1. A method, comprising:

determining, using image processing circuitry, whether one or more pixel intensity values of a pixel in a color space are within a pixel saturation range, wherein the pixel is one of a plurality of pixels of an input digital image;

setting, using the image processing circuitry, a luminance value of the pixel based on the determination of whether one or more pixel intensity values of the pixel are within the pixel saturation range;

determining, using the image processing circuitry, a pixel gain based on the set luminance value of the pixel;

applying, using the image processing circuitry, the determined pixel gain to the pixel; and

generating, using the image processing circuitry, an output digital image based on the input digital image, the generating the output digital image including determining and applying respective gains to the plurality of pixels of the input digital image, wherein when at least one of the one or more pixel intensity values is within the pixel saturation range, the setting the luminance value of the pixel is based on at least one of;

a pixel saturation intensity value; and

the at least one of the one or more pixel intensity values, wherein the setting the luminance value of the pixel based on the pixel saturation intensity value comprises applying a weight to the pixel saturation intensity value.

2. The method of claim 1 wherein luminance values of pixels of the output digital image are coded over a smaller number of bits than luminance values of pixels of the input digital image.

3. The method of claim 1, comprising:

when it is determined that the pixel intensity values of the pixel are outside the pixel saturation range, determining whether one or more pixel intensity values of the pixel are within a pixel black-out range; and

setting the luminance value of the pixel based on the determination of whether one or more pixel intensity values of the pixel are within the pixel black-out range.

4. The method of claim 3 wherein when it is determined that one or more of the pixel intensity values are within the pixel black-out range, the setting the luminance value of the pixel comprises applying a weight to a black-out intensity value.

5. The method of claim 3, comprising:

when it is determined that the pixel intensity values of the pixel are outside the pixel saturation range and outside the pixel black-out range, setting the luminance value of the pixel to a weighted combination of the one or more pixel intensity values of the pixel.

6. The method of claim 1 wherein the pixel is a macro-pixel and the determined gain is applied to all of the pixels of the macro-pixel.

7. The method of claim 1 wherein the color space is a Bayer color space.

8. A method, comprising:

determining, using image processing circuitry, whether one or more pixel intensity values of a pixel in a color space are within a pixel saturation range, wherein the pixel is one of a plurality of pixels of an input digital image;

setting, using the image processing circuitry, a luminance value of the pixel based on the determination of whether one or more pixel intensity values of the pixel are within the pixel saturation range, wherein when it is determined that the pixel intensity values of the pixel are outside the pixel saturation range, the luminance value of the pixel is set to a weighted combination of the one or more pixel intensity values;

determining, using the image processing circuitry, a pixel gain based on the set luminance value of the pixel;

applying, using the image processing circuitry, the determined pixel gain to the pixel; and

generating, using the image processing circuitry, an output digital image based on the input digital image, the generating the output digital image including determining and applying respective gains to the plurality of pixels of the input digital image.

9. The method of claim 8 wherein when at least one of the one or more pixel intensity values is within the pixel saturation range, the setting the luminance value of the pixel is based on at least one of:

a pixel saturation intensity value; and

the at least one of the one or more pixel intensity values.

10. The method of claim 9 wherein the setting the luminance value of the pixel based on the pixel saturation intensity value comprises applying a weight to the pixel saturation intensity value.

11. The method of claim 10, comprising determining the weight based on a difference between the at least one of the one or more pixel intensity values and the pixel saturation intensity value.

12. A device, comprising:

a memory; and

image processing circuitry, which, in operation;

determines whether one or more pixel intensity values of a pixel in a color space are within a pixel saturation range;

sets a luminance value of the pixel based on the determination of whether one or more pixel intensity values of the pixel are within the pixel saturation range;

determines a pixel gain based on the set luminance value of the pixel; and

applies the determined pixel gain to the pixel, wherein the pixel is one of a plurality of pixels of an input digital image, and the image processing circuitry, in operation, generates an output digital image based on the input digital image, the generating the output digital image including determining and applying respective gains to the plurality of pixels of the input digital image,

wherein when at least one of the one or more pixel intensity values is within the pixel saturation range, the setting the luminance value of the pixel is based on at least one of;

a pixel saturation intensity value; and

the at least one of the one or more pixel intensity values, and

wherein the setting the luminance value of the pixel based on the pixel saturation intensity value comprises applying a weight to the pixel saturation intensity value.

13. The device of claim 12 wherein the image processing circuitry, in operation,

when it is determined that the pixel intensity values of the pixel are outside the pixel saturation range, determines whether one or more pixel intensity values of the pixel are within a pixel black-out range;

and

sets the luminance value of the pixel based on the determination of whether one or more pixel intensity values of the pixel are within the pixel black-out range.

14. The device of claim 13 wherein the image processing circuitry, in operation,

when it is determined that the pixel intensity values of the pixel are outside the pixel saturation range and outside the pixel black-out range, sets the luminance value of the pixel to a weighted combination of the one or more pixel intensity values of the pixel.

15. The device of claim 12 wherein the pixel is a macro-pixel in a Bayer color space and the determined gain is applied to all of the pixels of the macro-pixel.

16. A system, comprising:

an image sensor, which, in operation, acquires digital images; and

image processing circuitry coupled to the image sensor, wherein the image processing circuitry, in operation;

determines whether one or more pixel intensity values of a pixel in a color space are within a pixel saturation range;

sets a luminance value of the pixel based on the determination of whether one or more pixel intensity values of the pixel are within the pixel saturation range;

determines a pixel gain based on the set luminance value of the pixel; and

applies the determined pixel gain to the pixel,

wherein the pixel is one of a plurality of pixels of a digital image acquired by the image sensor, and the image processing circuitry, in operation, generates an output digital image based on the acquired digital image, the generating the output digital image including determining and applying respective gains to the plurality of pixels of the acquired digital image, and

when at least one of the one or more pixel intensity values is within the pixel saturation range, the setting the luminance value of the pixel is based on a weight applied to one of;

a pixel saturation intensity value; and

the at least one of the one or more pixel intensity values.

17. A non-transitory computer-readable medium having executable instructions which configure image processing circuitry to perform a method, the method comprising:

determining whether one or more pixel intensity values of a pixel in a color space are within a pixel saturation range;

setting a luminance value of the pixel based on the determination of whether one or more pixel intensity values of the pixel are within the pixel saturation range;

determining, using the image processing circuitry, a pixel gain based on the set luminance value of the pixel; and

applying, using the image processing circuitry, the determined pixel gain to the pixel,

wherein the pixel is one of a plurality of macro-pixels of an input digital image and the method comprises generating an output digital image based on the input digital image, the generating the output digital image including determining and applying respective gains to the plurality of macro-pixels of the input digital image, and

when at least one of the one or more pixel intensity values is within the pixel saturation range, the setting the luminance value of the pixel is based on a weight applied to one of;

a pixel saturation intensity value; and

the at least one of the one or more pixel intensity values.