High-quality gradient-corrected linear interpolation for demosaicing of color images

• US 7,502,505 B2
• Filed: 03/15/2004
• Issued: 03/10/2009
• Est. Priority Date: 03/15/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Malvar, Henrique S., He, Li-wei, Cutler, Ross
• Primary Examiner(s): Le; Brian Q
• Application Number:
  US10/801,450
• Publication Number:
  US 20050201616A1
• Time in Patent Office: 1,821 Days
• Field of Search: 382/162, 382/167, 382/254, 382/299, 382/300, 348/246, 348/441, 358/525
• US Class Current: 382/162
• CPC Class Codes: G06T 3/4015 Image demosaicing, e.g. colour filter arrays [CFA] or Bayer patterns

Abstract

A gradient-corrected linear interpolation method and system for the demosaicing of color images. The method and system compute an interpolation using some a current technique (preferably a bilinear interpolation technique to reduce computational complexity), compute a correction term (such as a gradient of a desired color at a given pixel), and linearly combine the interpolation and the correction term to produce a corrected, high-quality interpolation of a missing color value at a pixel. The correction term may be a gradient correction term computed from the current color of the current pixel. This gradient is directly used to affect and correct the estimated color value produced by the prior art interpolation technique. The gradient-corrected linear interpolation method and system may also apply a gradient-correction gain to the gradient correction term. This gradient-correction gain affects the amount of gradient correction that is applied to the interpolation.

Claims

1. A method for interpolating a desired color at a current pixel in a color image, the current pixel having a current color, comprising:

using a computer to perform the following process actions;

computing an interpolation of the desired color at the current pixel using the desired color;

computing a gradient correction term using the current color;

determining a gradient-correction gain such that a mean-square error is minimized to produce an optimal gradient-correction gain;

applying the optimal gradient-correction gain to the gradient correction term to determine an amount of the gradient correction linearly combined with the interpolation; and

linearly combining the interpolation and the gradient correction term to obtain a corrected interpolation of the desired color at the current pixel.

2. The method as set forth in claim 1, further comprising using neighboring pixels of the desired color in computing the interpolation.

3. The method as set forth in claim 1, further comprising using the current pixel in computing the correction term.

4. The method as set forth in claim 3, further comprising using neighboring pixels of the current color in computing the correction term.

5. The method as set forth in claim 1, wherein the interpolation is a bilinear interpolation technique.

6. The method as set forth in claim 1, further comprising adding the interpolation and the correction term to obtain a corrected interpolation.

7. A computer-implemented method for interpolating a desired color at a current pixel in an image sensor, the current pixel having a first color, comprising:

using a computer to perform the following process actions;

computing a first interpolation of the desired color at the current pixel using pixels having the desired color;

computing a gradient correction using pixels having the first color;

linearly combining the first interpolation and the gradient correction to obtain a gradient-corrected interpolation of the desired color at the current pixel;

applying a gradient-correction gain to the gradient correction to affect the amount of the gradient correction that is linearly combined with the first interpolation; and

selecting the gradient-correction gain such that a mean-square error is minimized to produce an optimal gradient-correction gain.

8. The computer-implemented method of claim 7, further comprising adjusting the optimal gradient-correction gain to produce a simplified gradient-correction gain that allows computations using at least one of:

(a) integer arithmetic;

(b) no division operations.

9. The computer-implemented method of claim 7, wherein the first interpolation is a linear interpolation.

10. The computer-implemented method of claim 9, wherein the linear interpolation is a bilinear interpolation.

11. The computer-implemented method of claim 7, wherein the first interpolation is at least one of:

(a) a bilinear interpolation;

(b) a bi-cubic interpolation;

(c) a Lanczos interpolation.

12. The computer-implemented method of claim 7, further comprising:

defining a region of support as a size of a pixel neighborhood whose values are considered for computation associated with any given pixel;

selecting the region of support to include pixels nearest the current pixel having the first color; and

using the region of support to compute the first interpolation and the gradient correction.

13. The computer-implemented method of claim 12, wherein the region of support is a 5×

5 pixel region centered at the current pixel.

14. The computer-implemented method of claim 12, wherein the region of support is greater than a 5×

5 pixel region centered at the current pixel.

15. The computer-implemented method of claim 14, wherein the first interpolation is a nonlinear interpolation.

16. The computer-implemented method of claim 12, further comprising:

using a first region of support to compute the first interpolation; and

using a second region of support to compute the gradient correction.

17. The computer-implemented method of claim 16, wherein the first region of support is different from the second region of support.

18. A computer-readable medium having stored and encoded thereon computer-executable instructions for performing on the computer the computer-implemented method recited in claim 7.

19. A method for interpolating missing red-blue-green (RGB) data at a current pixel having a current color in a color image sensor, comprising:

using a computer to perform the following process actions;

using a first interpolation technique based on a missing color at the current pixel to determine a missing color estimate;

calculating a gradient correction based on the current color;

adjusting a gradient-correction gain based on characteristics of the color image sensor;

multiplying the gradient correction by the gradient-correction gain to obtain an adjusted gradient correction; and

combining in a linear manner the missing color estimate and the adjusted gradient correction to obtain a linearly corrected missing color estimate corresponding to at least some of the missing RGB data.

20. The method of claim 19, wherein the first interpolation technique is a bilinear interpolation.

21. The method of claim 19, wherein the gradient correction is a linear operator.

22. The method of claim 19, wherein the color image sensor is integrated into a digital camera system, and further comprising adjusting the gradient-correction gain based on characteristics of the digital camera system.

23. A process for linearly interpolating a missing color of a present pixel within a color image produced by a digital camera system having an image sensor, the present pixel having a first color, the process comprising:

using a computer to perform the following process actions;

defining a first region of support centered at the present pixel;

interpolating the missing color using an interpolation technique to obtain a first missing color estimation, the interpolation technique using pixels within the first region of support having the missing color;

defining a second region of support centered at the present pixel;

calculating a gradient correction using the present pixel and pixels within the second region of support having the first color;

applying a gradient-correction gain to the gradient correction that represents a percentage of the gradient correction to be used;

measuring global statistics of the color image;

varying the gradient-correction gain based on the global statistics; and

linearly combining the first missing color estimation and the gradient correction to obtain a gradient-corrected estimation of the missing color.

24. The process as set forth in claim 23, further comprising:

measuring local statistics for each region in the color image; and

varying the gradient-correction gain based on the local statistics.

25. The process as set forth in claim 23, further comprising computing the gradient-correction gain based on the missing color.

26. The process as set forth in claim 25, wherein the missing color is green, and further comprising setting the gradient-correction gain to a value of ½

.

27. The process as set forth in claim 25, wherein the missing color is red, and further comprising setting the gradient-correction gain to a value of ⅝

.

28. The process as set forth in claim 25, wherein the missing color is blue, and further comprising setting the gradient-correction gain to a value of ¾

.

29. The process as set forth in claim 23, wherein the first and second regions of support are a 5×

5 matrix of pixels.

30. A gradient-corrected linear interpolation system for interpolating a missing color value at a given pixel in a color image, the given pixel having a current color, comprising:

a general-purpose computer;

a computer-readable storage medium having stored and encoded thereon a computer program having program modules containing computer-executable instructions that are executable by the general-purpose computer, the computer program further comprising;

an interpolation module that computes an interpolation of the missing color value;

a correction term computation module that computes a gradient correction term for the interpolation, the correction term computation module further comprising;

a gradient-correction selector that determines a gradient-correction gain such that a mean-square error is minimized to produce an optimal gradient-correction gain and applies the optimal gradient-correction gain to the gradient correction term to determine an amount of gradient correction linearly combined with the interpolation; and

a linear combination module that linearly combines the interpolation and correction term to produce a corrected interpolation for the missing color value at the given pixel.

31. The gradient-corrected linear interpolation system as set forth in claim 30, wherein the correction term computation module further comprises a region of support module that selects a size of a region of support around the given pixel centered at the given pixel.

32. The gradient-corrected linear interpolation system as set forth in claim 30, wherein the correction term computation module further comprises a gradient correction module that computes a gradient correction using the given pixel and pixels in a region of support having the current color.