Single-step conversion from RGB Bayer pattern to YUV 4:2:0 format

• US 7,002,627 B1
• Filed: 06/19/2002
• Issued: 02/21/2006
• Est. Priority Date: 06/19/2002
• Status: Active Grant

• Current Assignee: Intellectual Ventures I LLC (Intellectual Ventures LLC)
• Original Assignee: Neomagic Corporation
• Inventors: Raffy, Philippe, Yassa, Fathy
• Primary Examiner(s): Groody, James J.
• Assistant Examiner(s): JELINEK, BRIAN J
• Application Number:
  US10/064,177
• Time in Patent Office: 1,343 Days
• Field of Search: 348/272, 348/273, 348/280, 348/441, 348/453, 348/252, 348/708, 345/604, 345/600, 345/589, 358/520
• US Class Current: 348/273
• CPC Class Codes: H04N 2209/046 Colour interpolation to calculate the missing colour values
• CPC Class Codes: H04N 23/84 for processing colour signals
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements

Abstract

Bayer-pattern pixels captured by an image sensor have only one of the three primary colors (RGB) per pixel location. Rather than interpolate the Bayer-pattern to generate the missing RGB color components for each pixel location, a direct conversion is performed to YUV pixels. A luminance calculator receives a 3×3 block of Bayer-pattern pixels and generates a luminance (Y) pixel for the center pixel location. Different coefficients are multiplied by each of the 9 Bayer-pattern pixels before summing to produce the center Y pixel, depending on the pattern location. A chrominance calculator first receives a 3×3 block of Y pixels generated by the luminance calculator. The 9 Y pixels are averaged to produce an average luminance. Two red or blue pixels in the 3×3 block are averaged and the average luminance subtracted. Then a constant is multiplied to generate the U and V pixels. Intermediate interpolated RGB avoided.

Claims

1. A direct converter comprising:

an input buffer coupled to receive a stream of pixels in a Bayer pattern wherein each pixel location has no more than one of a red (R) pixel, a blue (B) pixel, and a green (G) pixel;

a luminance calculator, coupled to receive an input block of the pixels from the input buffer, the input block including a plurality of green pixels and at least one blue pixel and at least one red pixel in the Bayer pattern, the luminance calculator generating from the green, red, and blue pixels in the input block a luminance pixel for a pixel location within the input block;

a luminance buffer for storing a plurality of luminance pixels generated by the luminance calculator including a luminance block of luminance pixels that has at least some pixel locations that correspond to pixel locations within the input block; and

a chrominance calculator, coupled to both the input buffer and to the luminance buffer, receiving at least two red pixels and at least two blue pixels within the input block, and receiving the luminance block of luminance pixels from the luminance buffer, the chrominance calculator calculating a first chrominance value from an average of the at least two blue pixels and from the luminance block of luminance pixels and a second chrominance value from an average of the at least two red pixels and from the luminance block of luminance pixels, whereby luminance and chrominance values are calculated directly from the red, green, and blue pixels in the Bayer pattern.

2. The direct converter of claim 1 wherein missing R, G, B color components in the Bayer pattern are not generated by interpolation but luminance and chrominance values are directly generated from the Bayer pattern without interpolation of R, G, B pixels to generate missing R, G, B pixels.

3. The direct converter of claim 2 wherein the chrominance calculator further comprises:

a luminance averager, receiving the luminance block of luminance pixels, for generating an average luminance of the luminance block;

a chrominance generator that receives the average luminance from the luminance averager and receives blue pixels from the input buffer, the chrominance generator combining the average luminance and the blue pixels to generate the first chrominance value, the chrominance generator receiving the average luminance and the red pixels from the input buffer and combining the average luminance and the red pixels to generate the second chrominance value,

whereby the average luminance is generated and used as an intermediate when generating the first and second chrominance values.

4. The direct converter of claim 3 wherein green pixels do not contribute to

the first chrominance value or to the second chrominance value except for contributing to the average luminance,

whereby the chrominance generator does not include contributions from green pixels in the input block but only from red or blue pixels from the input block.

5. The direct converter of claim 4 wherein the first chrominance value is a U pixel and the second chrominance value is a V pixel and the luminance pixel is a Y pixel in a YUV format.

6. The direct converter of claim 2 wherein the input block is at least a 3×

3 block of at least three rows and three columns of pixels in the Bayer pattern;

wherein the luminance pixel generated by the luminance calculator is at a central pixel location surrounded by other luminance pixels in the luminance block.

7. The direct converter of claim 6 wherein the luminance block is a same size as the input block.

8. The direct converter of claim 7 wherein the luminance block corresponds to same pixel locations as the input block.

9. The direct converter of claim 6 wherein the chrominance calculator is activated for fewer pixel locations that the luminance calculator;

wherein more luminance pixels are generated than first chrominance values and more luminance pixels are generated than second chrominance values.

10. The direct converter of claim 2 wherein the chrominance calculator is activated only when a central pixel location within the input block has a green pixel and not a red pixel and not a blue pixel.

11. The direct converter of claim 2 wherein the luminance calculator multiplies each pixel in the input block by a corresponding coefficient in a coefficient block to produce intermediate products;

wherein the luminance calculator sums the intermediate products to generate the luminance pixel.

12. The direct converter of claim 11 wherein the coefficient block is selected from a plurality of four coefficient blocks based on a pattern of the R, G, and B pixels in the input block.

13. A method for directly generating YUV pixels from red (R), green (G), blue (B) pixels in an un-interpolated pattern comprising:

receiving an input block of at least 3 rows of at least 3 pixels per row of R, G, and B pixels in the un-interpolated pattern wherein each pixel location in the un-interpolated pattern is a partial pixel that is missing at least one of the R, G, and B color components;

determining a pattern type for the input block and selecting a selected coefficient block in response to the pattern type;

multiplying the input block by the selected coefficient block and summing to generate a Y component that represents an average brightness at a center of the input block;

generating and storing Y components for each pixel location;

reading stored Y components for locations in the input block and generating an average Y value for the input block from Y components;

reading at least two B pixels from the input block;

generating a U component from the at least two B pixels and from the average Y value while ignoring R and G pixels from the input block;

reading at least two R pixels from the input block; and

generating a V component from the at least two R pixels and from the average Y value while ignoring B and G pixels from the input block;

wherein the U and V components represent color of a YUV pixel while the Y component represents brightness of the YUV pixel,

whereby R, G, B pixels in the un-interpolated pattern are directly converted to Y, U, and V components of YUV pixels without RGB interpolation.

14. The method of claim 13 wherein generating the U component and generating the V component occur when a center pixel in the input block is a G pixel, but do not occur when the center pixel is a R or a G pixel.

15. The method of claim 13 wherein the average Y value is an equal-weighted average of all Y components in pixel locations of the input block.

16. The method of claim 13 wherein the un-interpolated pattern is a Bayer pattern wherein each pixel location is a mono-color pixel that is missing two of the R, G, and B color components.

17. A color-space converter comprising:

input buffer means, receiving red (R), green (G), and blue (B) mono-color pixels arrayed in a pattern representing an image, for storing an input block of at least 3 lines of at least 3 mono-color pixels per line;

luminance calculator means, examining a pattern of the R, G, B pixels in the input block to determine a coefficient block, for multiplying the R, G and B pixels in the input block by the coefficient block to generate a luminance component for a center pixel location within the input block;

luminance storage means, receiving luminance components from the luminance calculator means, for storing luminance components for pixel locations in a YUV color space representing the image; and

chrominance calculator means, receiving at least two B pixels from the input block and receiving at least two R pixels from the input block, for generating a U chrominance component for the center pixel location within the input block by averaging the at least two B pixels and averaging at least 9 luminance components from the luminance storage means for pixel locations within the input block, and for generating a V chrominance component for the center pixel location within the input block by averaging the at least two R pixels and averaging at least 9 luminance components from the luminance storage means for pixel locations within the input block,

whereby Y, U, and V components are generated directly from the R, G, and B mono-color pixels in the input block without generation of multi-color RGB pixels.

18. The color-space converter of claim 17 wherein the luminance calculator means and the chrominance calculator means are programmable means in a digital-signal processor (DSP) or in an associative array processor.

19. The color-space converter of claim 17 wherein the input block is exactly 3 by 3 pixels and the center pixel location is a middle location.

20. The color-space converter of claim 17 wherein the chrominance calculator means includes difference means for generating a U difference and a V difference by subtracting an average of the at least 9 luminance components from an average of the at least two B pixels or the at least two R pixels;

wherein the chrominance calculator means further includes constant means for multiplying the U difference by a first constant to generate the U chrominance component, and for multiplying the V difference by a second constant to generate the U chrominance component.