Method and/or apparatus for implementing high dynamic range image processing in a video processing system

• US 9,426,381 B1
• Filed: 10/28/2015
• Issued: 08/23/2016
• Est. Priority Date: 06/28/2010
• Status: Active Grant

• Current Assignee: Ambarella International LP (Ambarella Incorporated)
• Original Assignee: Ambarella Incorporated
• Inventors: Kwok, Wilson, Kohn, Leslie D.
• Primary Examiner(s): Atala, Jamie
• Assistant Examiner(s): Mung, On S
• Application Number:
  US14/925,225
• Time in Patent Office: 300 Days
• Field of Search: 375/240.01, 375/240.25, 375/240.26
• US Class Current: 1/1
• CPC Class Codes: H04N 19/98 Adaptive-dynamic-range coding [ADRC]
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
• CPC Class Codes: H04N 25/57 Control of the dynamic range
• CPC Class Codes: H04N 7/12 Systems in which the television signal is transmitted via one channel or a plurality of parallel channels, the bandwidth of each channel being less than the bandwidth of the television signal H04N7/24 takes precedence
• CPC Class Codes: H04N 7/52 Systems for transmission of a pulse code modulated video signal with one or more other pulse code modulated signals, e.g. an audio signal or a synchronizing signal assembling of a multiplex stream by combining a video stream with other content or additional data, remultiplexing of multiplex streams, insertion of stuffing bits into the multiplex stream, assembling of a packetised elementary stream at server side H04N21/236; disassembling of a multiplex stream, remultiplexing of multiplex streams, extraction or processing of Service Information, disassembling of packetised elementary stream at client side H04N21/434
• CPC Class Codes: H04N 9/74 for obtaining special effects H04N9/65 - H04N9/73 take precedence

Abstract

An apparatus comprising a scaling circuit, a luma circuit and a blending circuit. The scaling circuit may generate a plurality of scaled frames in response to a first plurality of frames generated by a sensor. The first plurality of frames may have a first exposure. The luma circuit may generate an average luminance value for each of a plurality of processed pixels in each of a second of the plurality of frames generated by the sensor. The second of the plurality of frames may have a second exposure and each of the average luminance values is calculated based on a plurality of neighboring pixels in a neighborhood of the processed pixel.

Claims

1. An apparatus comprising:

a scaling circuit configured to generate a plurality of scaled frames each having a plurality of pixels with a first exposure in response to a first subset of a plurality of frames generated by a sensor;

a luma circuit configured to generate an average luminance value for each of a plurality of pixels having a second exposure in each of a second subset of said plurality of frames generated by said sensor; and

a blending circuit configured to generate a plurality of output frames, wherein (A) each of said pixels of said output frames are selected in response to one of said average luminance values and (B) portions of each of said output frames provide a gradual linear blend between said pixels having said first exposure and said pixels having said second exposure.

2. The apparatus according to claim 1, wherein each pixel of each of said plurality of output frames is generated by selecting (i) one of said pixels having said first exposure from one of said scaled frames or (ii) one of said pixels having said second exposure from one of said second subset of frames.

3. The apparatus according to claim 1, wherein each of said average luminance values is calculated based on a plurality of neighboring pixels in a neighborhood of a current one of said pixels having said second exposure.

4. The apparatus according to claim 3, wherein said plurality of neighboring pixels comprises a Bayer group of pixels.

5. The apparatus according to claim 3, wherein said neighborhood comprises an integral number of said neighboring pixels each having a ratio of two of a first type of pixel for every one of a second type of pixel and one of a third type of pixel.

6. The apparatus according to claim 5, wherein (i) said integral number of said neighboring pixels comprise a pixel quad, (ii) said first type of pixel comprises a green pixel, (iii) said second type of pixel comprises a red pixel and (iv) said third type of pixel comprises a blue pixel.

7. The apparatus according to claim 1, wherein said sensor alternately generates frames comprising one of said first subset of frames and one of said second subset of frames.

8. The apparatus according to claim 1, wherein said first subset of said plurality of frames and said second subset of said plurality of frames are generated in a Color Filter Array (CFA) domain.

9. The apparatus according to claim 1, further comprising:

a demosaic circuit configured to generate an output signal in response to said plurality of output frames, wherein said output signal comprises a video signal in a format for presentation by a display device.

10. The apparatus according to claim 1, wherein (i) said first exposure is longer than said second exposure and (ii) said first exposure emphasizes bright portions of said first subset of frames and said second exposure emphasizes dark portions of said second subset of frames.

11. The apparatus according to claim 1, wherein said plurality of output frames are implemented in real time.

12. The apparatus according to claim 1, wherein said apparatus is configured to implement high dynamic range image processing.

13. An apparatus comprising:

means for generating a plurality of scaled frames each having a plurality of pixels with a first exposure in response to a first subset of a plurality of frames generated by a sensor;

means for generating an average luminance value for each of a plurality of pixels having a second exposure in each of a second subset of said plurality of frames generated by said sensor; and

means for generating a plurality of output frames, wherein (A) each of said pixels of said output frames are selected in response to one of said average luminance values and (B) portions of each of said output frames provide a gradual linear blend between said pixels having said first exposure and said pixels having said second exposure.

14. A method for implementing high dynamic range image processing, comprising the steps of:

(A) generating a plurality of scaled frames each having a plurality of pixels with a first exposure in response to a first subset of a plurality of frames generated by a sensor;

(B) generating an average luminance value for each of a plurality of pixels having a second exposure in each of a second subset of said plurality of frames generated by said sensor; and

(C) generating a plurality of output frames, wherein (i) each of said pixels of said output frames are selected in response to one of said average luminance values and (ii) portions of each of said output frames provide a gradual linear blend between said pixels having said first exposure and said pixels having said second exposure.

15. The method according to claim 14, wherein said sensor alternately generates frames comprising one of said first subset of frames and one of said second subset of frames.

16. The method according to claim 14, wherein said first subset of said plurality of frames and said second subset of said plurality of frames are generated in a Color Filter Array (CFA) domain.

17. The method according to claim 14, further comprising the step of:

generating an output signal by demosaicing said plurality of output frames, wherein said output signal comprises a video signal in a format for presentation by a display device.

18. The method according to claim 14, wherein (i) said first exposure is longer than said second exposure and (ii) said first exposure emphasizes bright portions of said first subset of frames and said second exposure emphasizes dark portions of said second subset of frames.

19. The method according to claim 14, wherein said plurality of output frames are implemented in real time.