Adaptive image filter for filtering image information

• US 20070183681A1
• Filed: 02/09/2006
• Published: 08/09/2007
• Est. Priority Date: 02/09/2006
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Li, Hsiang-Tsun, Hung, Szepo
• Granted Patent: US 7,860,334 B2
• Time in Patent Office: Days
• US Class Current: 382/261
• CPC Class Codes: H04N 23/12 with one sensor only
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements
• CPC Class Codes: H04N 25/683 by defect estimation performed on the scene signal, e.g. real time or on the fly detection

Abstract

This disclosure describes adaptive filtering techniques to improve the quality of captured imagery, such as video or still images. In particular, this disclosure describes adaptive filtering techniques that filter each pixel as a function of a set of surrounding pixels. An adaptive image filter may compare image information associated with a pixel of interest to image information associated with a set of surrounding pixels by, for example, computing differences between the image information associated with the pixel of interest and each of the surrounding pixels of the set. The computed differences can be used in a variety of ways to filter image information of the pixel of interest. In some embodiments, for example, the adaptive image filter may include both a low pass component and high pass component that adjust as a function of the computed differences.

Claims

1. A method comprising:

computing differences between image information associated with a pixel of interest and image information associated with pixels that surround the pixel of interest;

determining a low pass component and a high pass component of a filter as a function of the computed differences; and

applying the filter to the image information associated with the pixel of interest.

2. The method of claim 1, wherein determining the low pass component of the filter comprises:

comparing each of the computed differences to a low pass threshold;

summing computed differences that are less than the low pass threshold to obtain a low pass component value; and

determining the low pass component based on the low pass component value.

3. The method of claim 2, wherein determining the low pass component of the filter further comprises dividing the low pass component value by a programmable value to determine the low pass component of the filter.

4. The method of claim 1, wherein determining the high pass component of the filter comprises:

comparing each of the computed differences to a high pass threshold; and

summing computed differences that are greater than the high pass threshold to obtain a high pass component value; and

determining the high pass component based on the high pass component value

5. The method of claim 4, wherein determining the high pass component of the filter further comprises dividing the high pass component value by a programmable value to determine the high pass component of the filter.

6. The method of claim 1, wherein applying the filter to the image information associated with the pixel of interest comprises filtering the image information according to the equation:

Out_pixel=in_pixel+(lpf_diff/(2ˆ

LPF_SHIFT))−

(hpf_diff/(2ˆ

HPF_SHIFT)),

where out_pixel is a value of the filtered image information, in_pixel is a value of the unfiltered image information, lpf_diff is a sum of computed differences that are less than a low pass threshold, hpf_diff is a sum of computed differences that are greater than a high pass threshold, and LPF_SHIFT and HPF_SHIFT are programmable variables.

7. The method of claim 1, further comprising:

buffering image information associated with the pixel of interest and the surrounding pixels; and

correcting image information associated with one or more defective pixels using the buffered image information,

wherein computing the differences between image information associated with the pixel of interest and image information associated with each of the surrounding pixels comprises computing the differences using the buffered image information.

8. The method of claim 1, wherein computing differences between image information associated with the pixel of interest and image information associated with each of the surrounding pixels comprises computing differences in intensity values associated with the pixels.

9. The method of claim 1, wherein the image information associated with the pixels represent only a single color.

10. The method of claim 1, wherein the image information associated with the pixels is in one of the RGB domain and the YCbCr domain.

11. The method of claim 1, wherein identifying the set of pixels that surround the pixel of interest comprises identifying an n by n array of pixels that surround the pixel of interest.

12. The method of claim 1, wherein the pixels that surround the pixel of interest utilize a same color filter.

13. A device comprising:

an image sensor array that captures image information, wherein the image sensor array includes a plurality of image sensors that capture image information associated with a plurality of pixels; and

an adaptive image filter that computes differences between image information associated with a pixel of interest and image information associated with pixels that surround the pixel of interest, determines a low pass component and a high pass component of a filter as a function of the computed differences, and applies the filter to the image information associated with the pixel of interest.

14. The device of claim 13, wherein the adaptive image filter compares each of the computed differences to a low pass threshold, sums computed differences that are less than the low pass threshold to obtain a low pass component value, and determines the low pass component based on the low pass component value.

15. The device of claim 16, wherein the adaptive image filter divides the low pass component value by a programmable value to determine the low pass component of the filter.

16. The device of claim 13, wherein the adaptive image filter compares each of the computed differences to a high pass threshold, sums computed differences that are greater than the high pass threshold to obtain a high pass component value, and determines the high pass component based on the high pass component value.

17. The device of claim 16, wherein the adaptive image filter divides the high pass component value by a programmable value to determine the high pass component of the filter.

18. The device of claim 13, wherein the adaptive image filter filters the image information associated with the pixel of interest according to the equation:

Out_pixel=in_pixel+(lpf_diff/(2ˆ

LPF_SHIFT))−

(hpf_diff/(2ˆ

HPF_SHIFT)),

where out_pixel is a value of the filtered image information, in_pixel is a value of the unfiltered image information, lpf_diff is a sum of computed differences that are less than a low pass threshold, hpf_diff is a sum of computed differences that are greater than a high pass threshold, and LPF_SHIFT and HPF_SHIFT are programmable variables.

19. The device of claim 13, further comprising a defective pixel correction unit that buffers image information associated with the pixel of interest and the set of surrounding pixels, and corrects image information associated with one or more defective pixels using the buffered image information, wherein the adaptive image filter computes the differences using the buffered image information.

20. The device of claim 13, wherein the adaptive image filter computes differences in intensity values associated with the pixels.

21. The device of claim 13, further comprising a plurality of color filters that cover the image sensors to capture image information representative of only a single color.

22. The device of claim 13, wherein the image information associated with the pixels is in one of the RGB domain and the YCbCr domain.

23. The device of claim 13, wherein the adaptive image filter identifies an n by n array of pixels that surround the pixel of interest.

24. The device of claim 13, wherein the adaptive image filter identifies surrounding pixels that utilize a same color filter.

25. A method comprising:

capturing image information for a plurality of pixels, wherein the image information associated with each of the pixels represents one of a plurality of colors;

selecting a pixel of interest and a set of surrounding pixels from the plurality of pixels, wherein the image information associated with the set of surrounding pixels and the pixel of interest represents the same color;

computing differences between image information associated with the pixel of interest and image information associated with each of the surrounding pixels of the set; and

filtering the image information associated with the pixel of interest as a function of the computed differences.

26. The method of claim 25, further comprising determining a low pass component and a high pass component of a filter as a function of the computed differences, wherein filtering the image information comprises filtering the image information associated with the pixel of interest in accordance with the filter.

27. The method of claim 25, wherein filtering the image information associated with the pixel of interest comprises filtering the image information according to the equation:

Out_pixel=in_pixel+(lpf_diff/(2ˆ

LPF_SHIFT))−

(hpf_diff/(2ˆ

HPF_SHIFT)),

where out_pixel is a value of the filtered image information, in_pixel is a value of the unfiltered image information, lpf_diff is a sum of computed differences that are less than a low pass threshold, hpf_diff is a sum of computed differences that are greater than a high pass threshold, and LPF_SHIFT and HPF_SHIFT are programmable variables.

28. A device comprising:

an image sensor array with a plurality of image sensors that capture image information associated with a plurality of pixels, wherein the image information associated with each of the pixels represents one of a plurality of colors; and

an adaptive image filter that selects a pixel of interest and a set of surrounding pixels from the plurality of pixels, computes differences between image information associated with the pixel of interest and image information associated with each of the surrounding pixels of the set, and filters the image information associated with the pixel of interest as a function of the computed differences, wherein the image information associated with the set of surrounding pixels and the pixel of interest represents the same color.

29. The device of claim 28, wherein the adaptive image filter computes a low pass component and a high pass component of the filter as a function of the computed differences.

30. The device of claim 28, wherein the adaptive image filter filters the image information associated with the pixel of interest according to the equation:

Out_pixel=in_pixel+(lpf_diff/(2ˆ

LPF_SHIFT))−

(hpf_diff/(2ˆ

HPF_SHIFT)),

where out_pixel is a value of the filtered image information, in_pixel is a value of the unfiltered image information, lpf_diff is a sum of computed differences that are less than a low pass threshold, hpf_diff is a sum of computed differences that are greater than a high pass threshold, and LPF_SHIFT and HPF_SHIFT are programmable variables.

31. A computer-readable medium comprising instructions to cause a processor to:

compute differences between image information associated with a pixel of interest and image information associated with pixels that surround the pixel of interest;

determine a low pass component and a high pass component of a filter as a function of the computed differences; and

apply the filter to the image information associated with the pixel of interest.

32. The computer-readable medium of claim 31, wherein determining the low pass component of the filter comprises:

comparing each of the computed differences to a low pass threshold;

summing computed differences that are less than the low pass threshold to obtain a low pass component value; and

determining the low pass component based on the low pass component value.

33. The computer-readable medium of claim 32, wherein the instructions cause the processor to divide the low pass component value by a programmable value to determine the low pass component of the filter.

34. The computer-readable medium of claim 32, wherein the instructions cause the processor to:

compare each of the computed differences to a high pass threshold; and

sum computed differences that are greater than the high pass threshold to obtain a high pass component value; and

determine the high pass component based on the high pass component value

35. The computer-readable medium of claim 34, wherein the instructions cause the processor to divide the high pass component value by a programmable value to determine the high pass component of the filter.

36. The computer-readable medium of claim 31, wherein the instructions cause the processor to filter the image information according to the equation:

Out_pixel=in_pixel+(lpf_diff/(2ˆ

LPF_SHIFT))−

(hpf_diff/(2ˆ

HPF_SHIFT)),

where out_pixel is a value of the filtered image information, in A pixel is a value of the unfiltered image information, lpf_diff is a sum of computed differences that are less than a low pass threshold, hpf_diff is a sum of computed differences that are greater than a high pass threshold, and LPF_SHIFT and HPF_SHIFT are programmable variables.

37. The computer-readable medium of claim 31, wherein the instructions cause the processor to:

buffer image information associated with the pixel of interest and the surrounding pixels; and

correct image information associated with one or more defective pixels using the buffered image information,

wherein computing the differences between image information associated with the pixel of interest and image information associated with each of the surrounding pixels comprises computing the differences using the buffered image information.

38. The computer-readable medium of claim 31, wherein the instructions cause the processor to compute differences in intensity values associated with the pixels to computing differences between image information associated with the pixel of interest and image information associated with each of the surrounding pixels.

39. The computer-readable medium of claim 31, wherein the image information associated with the pixels represent only a single color.

40. The computer-readable medium of claim 31, wherein the image information associated with the pixels is in one of the RGB domain and the YCbCr domain.

41. The computer-readable medium of claim 31, wherein the instructions cause the processor to identify an n by n array of pixels that surround the pixel of interest.

42. The computer-readable medium of claim 31, wherein the pixels that surround the pixel of interest utilize a same color filter.