HDR ENHANCEMENT WITH TEMPORAL MULTIPLEX

• US 20180308269A1
• Filed: 06/05/2017
• Published: 10/25/2018
• Est. Priority Date: 04/24/2017
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Baran, Stanley J., Appu, Abhishek R., Lee, Sang-Hee, Mohammed, Atthar H., Oh, Jong Dae, Chan, Hiu-Fai R., Ray, Joydeep, Parikh, Kunjal, Wang, Changliang, Kambhatla, Srikanth, Smith, Gary, Avadhanam, Satyanarayana, Hicks, Richmond, Johnston, Robert J., Biswal, Narayan, Bhattacharjee, Susanta
• Granted Patent: US 10,643,358 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20208 High dynamic range [HDR] image processing
• CPC Class Codes: G06T 5/92 based on global image properties
• CPC Class Codes: H04N 9/77 Circuits for processing the brightness signal and the chrominance signal relative to each other, e.g. adjusting the phase of the brightness signal relative to the colour signal, correcting differential gain or differential phase circuits for matrixing H04N9/67

Abstract

Systems, apparatuses and methods may a performance-enhanced computing system comprising a sensor for measuring luminance values corresponding to light focused onto the sensor at a plurality of pixel locations, a memory including a set of instructions, and a processor. The processor executes a set of instructions causing the system to generate a multi-segment tone mapping curve, generate a set of tone mapping values corresponding to the multi-segment tone mapping curve for equally spaced input values between zero and one for storage into a look up table, and process the luminance values using the look up table to apply the tone mapping curve to the luminance values of the pixels.

Claims

1. A system comprising:

a sensor for measuring luminance values corresponding to light focused onto the sensor at a plurality of pixel locations of a captured image, the luminance values having a dynamic range;

a memory including a set of instructions; and

a processor, wherein when executed by the processor, the instructions cause the system to;

generate a multi-segment tone mapping curve;

generate a set of tone mapping values corresponding to the multi-segment tone mapping curve for equally spaced input values between zero and one for storage into a look up table; and

process the luminance values using the look up table to apply the tone mapping curve to the luminance values of the pixels.

2. The system of claim 1, wherein the generation of the multi-segment tone mapping curve comprise:

setting initial values for a set of end pivot points on the multi-segment tone mapping curve;

computing additional pivot points on the multi-segment tone mapping curve between the set of end points to compress the dynamic range of the luminance values to a desired dynamic range;

mapping the tone mapping curve to values in the look up table at equally spaced locations; and

apply a multi-tap sync filter to the values within the look up table.

3. The system of claim 2, wherein the multi-tap sync filter comprises an exponential curve fitting function.

4. The system of claim 3, wherein the generation of the tone mapping curve creates a one to one line applying no tone mapping to the luminance values when the dynamic range is greater than a desired dynamic range.

5. The system of claim 4, wherein the desired dynamic range corresponds to a dynamic range for an output display device used to display the captured image.

6. The system of claim 1, wherein the sensor comprises a digital imaging device.

7. A semiconductor package apparatus comprising:

a substrate; and

logic coupled to the substrate, wherein the logic is implemented in one or more of configurable logic or fixed-functionality hardware logic, the logic to;

periodically scan a plurality of detection images from the sensor while the sensor remains enabled;

utilize a plurality of exposure estimates for each of the detection images to determine a plurality of shutter times; and

scan a plurality of capture images from the sensor for storage into a plurality of image buffers;

wherein the capture images include at least an under-exposed image, a well exposed image, and an over-exposed image.

8. The apparatus of claim 7, wherein the method further comprises combining the capture images into a single high dynamic range (HDR) image.

9. The apparatus of claim 7, wherein the logic further enables a sensor to begin collecting image pixel data corresponding to luminance of light focused on the sensor before beginning the scanning of the plurality of capture images, the scanning the plurality of capture images being scanned at each of the plurality of shutter times.

10. The apparatus of claim 9, wherein the plurality of detection images utilized to generate the plurality of shutter times corresponds to a plurality of capture images from a previous HDR image.

11. The apparatus of any one of claims 7 to 10, wherein the determining the exposure estimate for each of the detection images comprises counting the number of pixel values greater than a plurality predetermined threshold values.

12. The apparatus of claim 11, wherein the plurality of predetermined threshold values includes a saturated threshold value corresponding to a maximum pixel value generated by the sensor.

13. A method of operating a semiconductor package apparatus, comprising:

periodically scanning a plurality of detection images from the sensor while the sensor remains enabled;

utilizing a plurality of exposure estimates for each of the detection images to determine a plurality of shutter times; and

scanning a plurality of capture images from the sensor for storage into a plurality of image buffers;

wherein the capture images include at least an under-exposed image, a well exposed image, and an over-exposed image.

14. The method of claim 13, wherein the method further comprising combining the capture images into a single high dynamic range (HDR) image.

15. The method of claim 13, wherein the method further comprising enabling a sensor to begin collecting image pixel data corresponding to luminance of light focused on the sensor before beginning the scanning of the plurality of capture images, the scanning the plurality of capture images being scanned at each of the plurality of shutter times.

16. The method of claim 15, wherein the plurality of detection images utilized to generate the plurality of shutter times corresponds to a plurality of capture images from a previous HDR image.

17. The method of any one of claims 13 to 16, wherein the determining the exposure estimate for each of the detection images comprises counting the number of pixel values greater than a plurality predetermined threshold values.

18. The method of claim 17, wherein the plurality of predetermined threshold values includes a saturated threshold value corresponding to a maximum pixel value generated by the sensor.

19. At least one computer readable storage medium comprising a set of instructions, which when executed by a computing system, cause the computing system to:

generating a multi-segment tone mapping curve;

generating a set of tone mapping values corresponding to the multi-segment tone mapping curve for equally spaced input values between zero and one for storage into a look up table; and

processing the luminance values using the look up table to apply the tone mapping curve to the luminance values of the pixels.

20. The least one computer readable storage medium of claim 19, wherein the generation of the multi-segment tone mapping curve comprise:

setting initial values for a set of end pivot points on the multi-segment tone mapping curve;

computing additional pivot points on the multi-segment tone mapping curve between the set of end points to compress the dynamic range of the luminance values to a desired dynamic range;

mapping the tone mapping curve to values in the look up table at equally spaced locations; and

applying a multi-tap sync filter to the values within the look up table.

21. The least one computer readable storage medium of claim 20, wherein the multi-tap sync filter comprises an exponential curve fitting function.

22. The least one computer readable storage medium of claim 21, wherein the generation of the tone mapping curve creates a one to one line applying no tone mapping to the luminance values when the dynamic range is greater than a desired dynamic range.

23. The least one computer readable storage medium of claim 22, wherein the desired dynamic range corresponds to a dynamic range for an output display device used to display the captured image.

24. The least one computer readable storage medium of claim 29, wherein the sensor comprises a digital imaging device.