IMAGE PROCESSING FOR WIDE DYNAMIC RANGE (WDR) SENSOR DATA

• US 20170223267A1
• Filed: 06/15/2016
• Published: 08/03/2017
• Est. Priority Date: 02/03/2016
• Status: Active Grant

• Current Assignee: Texas Instruments, Inc.
• Original Assignee: Texas Instruments, Inc.
• Inventors: DABRAL, SHASHANK, MODY, MIHIR NARENDRA, HUA, GANG, LELL, ANTHONY, NANDAN, NIRAJ
• Granted Patent: US 9,871,965 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 2207/20208 High dynamic range [HDR] image processing
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: H04N 2201/0084 Digital still camera
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
• CPC Class Codes: H04N 23/743 Bracketing, i.e. taking a series of images with varying exposure conditions
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 25/587 acquired sequentially, e.g. using the combination of odd and even image fields
• CPC Class Codes: H04N 25/611 Correction of chromatic aberration
• CPC Class Codes: H04N 25/683 by defect estimation performed on the scene signal, e.g. real time or on the fly detection
• CPC Class Codes: H04N 5/265 Mixing

Abstract

A signal processing chain implements wide dynamic range (WDR) multi-frame processing including receiving raw image signals from a WDR sensor including a plurality of frames including a first frame including first exposure time pixel data and a second frame including second exposure time pixel data. Statistics for camera control are generated including first statistics for the first pixel data and second statistics for the second pixel data. The first and second pixel data are merged using WDR merge algorithm in a WDR merge block which utilizes the first and second statistics to generate a raw higher bit width single frame image. The single frame image is post-processed in post-processing block using at least a defect pixel correction algorithm, and at least a portion of tone mapping is performed on the single frame image after the post-processing to provide an output toned mapped image.

Claims

1. A method of wide dynamic range (WDR) multi-frame processing, comprising:

receiving raw image signals from a WDR sensor comprising a plurality of frames that each include a plurality of pixels, said plurality of frames including a first frame having first exposure time pixel data (first pixel data) and at least a second frame having second exposure time pixel data (second pixel data), said first and said second pixel data having an intensity value;

generating statistics for camera control including first statistics for said first pixel data and second statistics for said second pixel data;

merging said first pixel data and said second pixel data using a WDR merge algorithm which utilizes said first statistics and said second statistics to generate a raw higher bit width single frame image;

post-processing said raw higher bit width single frame image using at least a defect pixel correction algorithm, and

performing at least a portion of tone mapping to said raw higher bit width single frame image after said post-processing to provide an output toned mapped image.

2. The method of claim 1, wherein said first statistics and said second statistics are generated using time division multiplexing (TDM) across said first pixel data in said first frame and said second pixel data in said second frame.

3. The method of claim 1, further comprising before said post-processing performing an initial global tone mapping using a non-linear (NL) LUT that has a finer step size at lower pixel intensities compared to a step size at higher pixel intensities to reduce a pixel bit width from a pixel bit width in said raw higher bit width single frame image.

4. The method of claim 1, wherein said post-processing further comprises adaptive defective pixel correction (DPC), lens shading correction (LSC), and temporal multiplexed H3A.

5. The method of claim 1, wherein said plurality of frames further comprises a third frame having third exposure time pixel data (third pixel data), wherein said generating said statistics for camera control further includes generating third statistics for said third pixel data, and wherein said merging merges said first pixel data, said second pixel data and said third pixel data.

6. The method of claim 1, wherein said merging increases a bit width by at least 25%.

7. The method of claim 1, further comprising using said statistics to compute white balance (WB) which is fedback to a WDR merge block performing said merging.

8. The method of claim 3, wherein said NL LUT comprises a plurality of NL LUTs having different tone mapping curves covering different levels of said intensities further comprising switching between respective ones of said plurality of NL LUTs, and wherein software updates a content of one of said NL LUTs while another of said plurality of NL LUTs is in use.

9. A signal processing chain for wide dynamic range (WDR) multi-frame processing, comprising:

a statistics generation block;

a WDR merge block having a WDR merge algorithm, both said statistics generation block coupled for receiving raw image signals from a WDR sensor comprising a plurality of frames that each include a plurality of pixels, said plurality of frames including a first frame having first exposure time pixel data (first pixel data) and at least a second frame having second exposure time pixel data (second pixel data), said first and said second pixel data having an intensity value;

said statistics generation block for generating statistics for camera control including first statistics for said first pixel data and second statistics for said second pixel data;

said WDR merge block for merging said first pixel data and said second pixel data using said WDR merge algorithm which utilizes said first statistics and said second statistics to generate a raw higher bit width single frame image;

a post-processing block for post-processing said raw higher bit width single frame image using at least a defect pixel correction algorithm, and

a tone mapping block for performing at least a portion of tone mapping performed by said signal processing chain to provide an output tone mapped image.

10. The signal processing chain of claim 9, wherein said first statistics and said second statistics are generated by said statistics generation block using time division multiplexing (TDM) across said first pixel data in said first frame and said second pixel data in said second frame.

11. The signal processing chain of claim 9, further comprising a non-linear (NL) LUT that has a finer step size at lower pixel intensities compared to a step size at higher pixel intensities to reduce a pixel bit width from a pixel bit width in said raw higher bit width single frame image.

12. The signal processing chain of claim 9, wherein said post-processing further comprises adaptive defective pixel correction (DPC), lens shading correction (LSC), and temporal multiplexed H3A.

13. The signal processing chain of claim 9, wherein said plurality of frames further comprises a third frame having third exposure time pixel data (third pixel data), wherein said generating said statistics for camera control further includes generating third statistics for said third pixel data, and wherein said merging merges said first pixel data, said second pixel data and said third pixel data.

14. The signal processing chain of claim 9, wherein said merging increases a bit width by at least 25%.

15. The signal processing chain of claim 9, further comprising a processing block that uses said statistics to compute white balance (WB) which is fedback to said WDR merge block.

16. The signal processing chain of claim 11, wherein said NL LUT comprises a plurality of NL LUTs having different levels of said intensities further comprising switching between respective ones of said plurality of NL LUTs, and wherein software updates a content of one of said NL LUTs while another of said plurality of NL LUTs is in use.