Image interpolation circuit architecture and method for fast bi-cubic interpolation of image information

• US 6,300,935 B1
• Filed: 04/20/1999
• Issued: 10/09/2001
• Est. Priority Date: 04/20/1999
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Agilent Technologies, Inc.
• Inventors: Sachs, Todd S., Sobel, Arthur
• Primary Examiner(s): Brier, Jeffery
• Assistant Examiner(s): HAVAN, THU THAO
• Application Number:
  US09/295,727
• Time in Patent Office: 903 Days
• Field of Search: 345/149, 345/147, 345/150, 345/152, 345/153, 345/154, 345/155, 345/72, 345/76, 345/77, 345/132, 345/505, 345/519
• US Class Current: 345/589
• CPC Class Codes: G06T 3/4007 based on interpolation, e.g. bilinear interpolation image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403

Abstract

A circuit architecture and method are provided for interpolating a first color value associated with a first color and a second color value associated with a second color for use in generating a pixel that represents a portion of a digital image, based on a third color value that is associated with a third color. Pixel data generated by a digital image sensor is serially received at a register array organized in rows and columns that correspond to pixels of interest that are used in a bicubic interpolation process. Values stored in registers of the register array are coupled to and continuously available to four (4) dot product modules and an interpolator. As the serial data arrives, it is clocked stepwise through the registers, and concurrently used by the dot product modules and interpolator to compute the first color value and the second color value. Data that reaches the end of a line of registers is moved into a corresponding shift register for temporary storage until it is needed again. Advantageously, the method may be implemented in integrated circuit hardware and using fast combinational logic with no CPU multiply operations and no floating-point operations. A particular application is in interpolating complementary colors for pixel information received from a color area sensor, such as a CCD image sensor, in a digital camera.

Claims

1. An image processing circuit that generates a plurality of color values each representing color components of a pixel of a digital image, the circuit comprising:

a register array that is organized in a plurality of lines of registers and that represents a portion of an array of pixels of the digital image;

a plurality of shift registers, in which each of the shift registers is coupled to receive an input from one of the lines of registers and to shift said input to another one of the lines of registers;

a plurality of dot product modules, each coupled to the register array, each of which generates an interpolated color value based on the register array; and

an interpolator module configured to receive image data values stored in the register array into the dot product modules, select at least one of the dot product modules for use in generating one of the color values, and generate the color values based on at least a result of the selected dot product module.

2. A circuit as recited in claim 1, wherein the interpolator module is further configured to generate one of the color values based on a center register value that is stored in a center register of the register array.

3. A circuit as recited in claim 1, wherein the interpolator module receives one or more sensor size signals that identify one or more attributes of a sensor that is used to form the digital image, and wherein the interpolator module selects one of the dot product modules for use in generating one of the color values based on a current relationship of information that is stored in the register array to the attributes.

4. A circuit as recited in claim 1, wherein the interpolator module receives a height signal and a width signal that respectively identify dimensions of a sensor that is used to form the digital image, and wherein the interpolator module selects one of the dot product modules for use in generating one of the color values based on a current relationship of information that is stored in the register array to the dimensions.

5. A circuit as recited in claim 1, wherein the register array comprises forty-nine (49) registers organized in seven (7) lines of seven (7) registers each.

6. A circuit as recited in claim 5, wherein the dot product modules comprise four (4) dot product modules, in which each of the dot product modules generates an interpolated color value of a first pre-determined color based on a center value of the register array that represents a second color.

7. A circuit as recited in claim 1, wherein the dot product modules comprise four (4) dot product modules, in which each of the dot product modules generates an interpolated color value of a first pre-determined color based on a center value of the register array that represents a second color.

8. A circuit as recited in claim 1, in which each of the dot product modules generates an interpolated color value of a first predetermined color based on a center value of the register array that represents a second color, and wherein the dot product modules comprise:

a first dot product module that generates a Green color value when the center value represents a Blue value or a Red value;

a second dot product module that generates a Red color value when the center value represents a Blue value and generates a Blue value when the center value represents a Red value;

a third dot product module that generates a Red color value when the center value represents a Green value among a plurality of alternating Green, Blue values and generates a Blue value when the center value represents a Green value among a plurality of alternating Blue, Green values; and

a fourth dot product module that generates a Red color value when the center value represents a Green value among a plurality of alternating Green, Blue values and generates a Blue value when the center value represents a Green value among a plurality of alternating Blue, Green values.

9. A circuit as recited in claim 8, wherein the first dot product module is configured to generate an output color value M1 based on input pixel values at positions P12, P14, P16, P21, P23, P25, P27, P32, P34, P36, P41, P43, P45, P47, P52, P54, P56, P61, P63, P65, P67, P72, P74, P76 of the register array, according to the logic:

 

M1=((P23+P25+P32+P36+P52+P56+P63+P65)<

<

4+(P34+P43+P45+P54)<

<

6

10. A circuit as recited in claim 8, wherein the second dot product module is configured to generate an output color value M2 based on input pixel values at positions P11, P13, P15, P17, P31, P33, P35, P37, P51, P53, P55, P57, P71, P73, P75, and P77, of the register array, according to the logic:

11. A circuit as recited in claim 8, wherein the third dot product module is configured to generate an output color value M3 based on input pixel values at positions values P12, P14, P16, P32, P34, P36, P52, P54, P56, P72, P74, and P76 of the register array, according to the logic:

12. A circuit as recited in claim 8, wherein the fourth dot product module is configured to generate an output color value M4 based on input pixel values at positions P21, P23, P25, P27, P41, P43, P45, P47, P61, P63, P65, P67 of the register array, according to the logic:

13. A method of interpolating a first color value associated with a first color and a second color value associated with a second color for use in generating a current pixel of interest that represents a portion of a digital image, wherein the method is carried out using an image processing circuit comprises a register array organized in a plurality of lines of registers;

a plurality of shift registers, in which each of the shift registers is coupled to receive an input from one of the lines of registers and to shift said input to another one of the lines of registers;

a plurality of dot product modules, each coupled to the register array, each of which generates an interpolated color value based on the register array; and

an interpolator module configured to receive image data values stored in the register array into the dot product modules, select at least one of the dot product modules for use in generating one of the color values, and generate the color values based on at least a result of the selected dot product module, the method comprising the steps of;

creating and storing, in one of the dot product modules, first coefficient values that are associated with the first color and are associated with other values of the first color for pixels that surround the current pixel of interest;

bitwise shifting each of the first coefficient values by a predetermined power of 2;

creating and storing the first color value by applying the first coefficient values to the other values of the first color for the pixels that surround the current pixel of interest, using a bitwise shift operation;

bitwise shifting the first color value by a complement of the pre-determined power of 2; and

displaying the digital image using at least the pixel comprising the first color value.

14. A method of claim 13, further including the steps of:

creating and storing, in one of the dot product modules, second coefficient values that are associated with the second color value and are associated with other values of the second color for other pixels that surround the current pixel of interest; and

bitwise shifting each of the first coefficient values and the second coefficient values by a pre-determined power of 2.

15. A method of claim 14, further including the step of selecting the pre-determined power of 2 from among the values (64, 128, 256, 512).

16. A method of claim 14, wherein the steps of creating and storing a first matrix of first coefficient values and bitwise shifting each of the first coefficient values by a pre-determined power of 2 comprise the steps of:

creating and storing a first matrix of first coefficient values that are associated with the first color value and in which each of the first coefficient values is a multiple of a power of 2.

17. A method of claim 14, wherein the steps of creating and storing a first matrix of first coefficient values that are associated with the first color value and bitwise shifting each of the first coefficient values by a pre-determined power of 2 comprise the steps of:

creating and storing a first matrix of first coefficient values that are associated with the first color value and in which each of the first coefficient values is a multiple of a power of 2 selected from among the range (2, 4, 16, 64, 128, 256,

     512).

18. A method as recited in claim 13, wherein the steps of creating and storing the first color value comprise the steps of:

adding all non-zero positive first coefficient values that are associated with the first color to create and store a positive sum value;

adding all non-zero negative first coefficient values that are associated with the first color to create and store a negative sum value;

subtracting the negative sum value from the positive sum value to create and store a net value; and

bitwise shifting the first color value by a number of bits equal to log₂ of the net value and storing a result as the first color value.

19. An image processing circuit that generates a plurality of color values each representing color components of a pixel of a digital image, the circuit comprising:

a digital storage array organized in a plurality of lines, each of the lines comprising a plurality of digital storage elements, each of which may receive a pixel value;

a plurality of shift registers, in which each of the shift registers is coupled to receive an input from one of the lines of the digital storage array and to shift said input to a first digital storage element of another one of the lines;

a plurality of computing modules, each coupled to the digital storage array, each of which generates an interpolated color value based on values stored in the digital storage array; and

an interpolator module configured to receive image data values stored in the digital storage array into the computing modules, select at least one of the computing modules for use in generating one of the color values, and generate the color values based on at least a result of the selected computing modules.

20. A digital image processing system capable of interpolating a first color value associated with a first color and a second color value associated with a second color for use in generating a current pixel of interest that represents a portion of a digital image, the system comprising:

an image processing circuit comprising;

a register array organized in a plurality of lines of registers;

a plurality of shift registers, in which each of the shift registers is coupled to receive an input from one of the lines of registers and to shift said input to another one of the lines of registers;

a plurality of dot product modules, each coupled to the register array, each of which generates an interpolated color value based on the register array; and

an interpolator module configured to receive image data values stored in the register array into the dot product modules, select at least one of the dot product modules for use in generating one of the color values, and generate the color values based on at least a result of the selected dot product module; and

a computer-readable medium coupled to the interpolator module and comprising one or more instructions which may cause the interpolator module to carry out the steps of;

creating and storing, in one of the dot product modules, first coefficient values that are associated with the first color and are associated with other values of the first color for pixels that surround the current pixel of interest;

bitwise shifting each of the first coefficient values by a pre-determined power of 2;

creating and storing the first color value by applying the first coefficient values to the other values of the first color for the pixels that surround the current pixel of interest, using a bitwise shift operation;

bitwise shifting the first color value by a complement of the pre-determined power of 2; and

displaying the digital image using at least the pixel comprising the first color value.

21. A system as recited in claim 20, wherein the instructions may cause the interpolator module to further carry out the steps of:

creating and storing, in one of the dot product modules, second coefficient values that are associated with the second color value and are associated with other values of the second color for other pixels that surround the current pixel of interest; and

bitwise shifting each of the first coefficient values and the second coefficient values by a pre-determined power of 2.

22. A system as recited in claim 21, wherein the instructions may cause the interpolator module to further carry out the steps of selecting the pre-determined power of 2 from among the values (64, 128, 256, 512).

23. A system as recited in claim 21, wherein the steps of creating and storing a first matrix of first coefficient values and bitwise shifting each of the first coefficient values by a predetermined power of 2 comprise the steps of:

creating and storing a first matrix of first coefficient values that are associated with the first color value and in which each of the first coefficient values is a multiple of a power of 2.

24. A system as recited in claim 21, wherein the steps of creating and storing a first matrix of first coefficient values that are associated with the first color value and bitwise shifting each of the first coefficient values by a pre-determined power of 2 comprise the steps of:

creating and storing a first matrix of first coefficient values that are associated with the first color value and in which each of the first coefficient values is a multiple of a power of 2 selected from among the range (2, 4, 16, 64, 128, 256,

     512).

25. A system as recited in claim 20, wherein the steps of creating and storing the first color value comprise the steps of:

adding all non-zero positive first coefficient values that are associated with the first color to create and store a positive sum value;

adding all non-zero negative first coefficient values that are associated with the first color to create and store a negative sum value;

subtracting the negative sum value from the positive sum value to create and store a net value; and

bitwise shifting the first color value by a number of bits equal to log₂ of the net value and storing a result as the first color value.