TECHNIQUES FOR ADJUSTING THE EFFECT OF APPLYING KERNALS TO SIGNALS TO ACHIEVE DESIRED EFFECT ON SIGNAL

• US 20080266413A1
• Filed: 01/07/2008
• Published: 10/30/2008
• Est. Priority Date: 04/24/2007
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Inventors: Cohen, Noy, Shabtay, Gal, David, Ya'ara, Mendlovic, David, Goldenberg, Ephraim Robert
• Granted Patent: US 8,306,348 B2
• Time in Patent Office: Days
• US Class Current: 348/222.1
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 5/002 Denoising; Smoothing noise processing or correction adapted to be used in an image pickup device containing and electronic image sensor H04N23/81, H04N25/60 - H04N25/67
• CPC Class Codes: G06T 5/003 Deblurring; Sharpening vibration or motion blur correction for cameras comprising an electronic image sensor H04N23/682
• CPC Class Codes: G06T 5/008 Local, e.g. shadow enhancement
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements
• CPC Class Codes: H04N 25/61 the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"

Abstract

A method and apparatus for adjusting the effect of applying a kernel to a signal is provided. The adjustment may be based on the magnitude of an effect that a kernel matrix is intended to have on the signal matrix. The adjustments can be based on factors including, but not limited to, a signal-to-noise ratio of the signal data, properties of a device (e.g., lens) used to capture the signal data, or a metric that is derived based on an analysis of the signal data. Processing image data in accordance with an embodiment of the present invention is aimed at regaining lost contrast that may be due to properties of optics that are used to capture the image data.

Claims

1. A method of processing a signal, said method comprising the steps of:

for each signal matrix of a plurality of signal matrices, wherein each signal matrix contains entries that describe a different portion of the signal;

based on the magnitude of an effect that a kernel matrix is intended to have on the signal matrix, adjusting the magnitude of the effect that is achieved due to processing the signal matrix based on the kernel matrix.

2. The method of claim 1, wherein adjusting the magnitude of the effect that is achieved due to processing the signal matrix based on the kernel matrix includes:

generating a modified kernel matrix for the signal matrix; and

applying the modified kernel matrix to the signal matrix to determine a new value for a first entry for the signal matrix.

3. The method of claim 1, wherein adjusting the magnitude of the effect that is achieved due to processing the signal matrix based on the kernel matrix includes:

applying the kernel matrix to the signal matrix to generate an intermediate pixel value; and

determining a weighted average of the intermediate pixel value and a first entry of the signal matrix.

4. The method of claim 1, wherein the effect of applying the kernel matrix enhances image contrast in the signal.

5. The method of claim 1, wherein the effect of applying the kernel matrix performs de-noising on the image in the signal.

6. The method of claim 1, further comprising determining the magnitude of the effect that the kernel matrix is intended to have on the signal matrix based on a signal-to-noise level associated with the signal.

7. The method of claim 1, further comprising determining the magnitude of the effect that the kernel matrix is intended to have on the signal matrix based on properties of a lens that is used to capture the signal described by the signal matrices.

8. The method of claim 1, further comprising determining the magnitude of the effect that the kernel matrix is intended to have on the signal matrix based on characteristics of data in the signal matrix.

9. The method of claim 1, wherein determining the magnitude of the effect that the kernel matrix is intended to have on the signal matrix is based on a location of a pixel within an image sensor.

10. The method of claim 1, further comprising, prior to processing the signal matrix based on the kernel matrix, modifying the kernel matrix based on a motion blur characteristic of the image data.

11. An apparatus comprising:

a lens;

an image capture device that is optically coupled to the lens, wherein the image capture device captures an image; and

logic that is configured to process the image based on a kernel matrix, wherein the logic is operable to;

for each signal matrix of a plurality of signal matrices, wherein each signal matrix contains entries that describe a different portion of the signal;

based on the magnitude of an effect that a kernel matrix is intended to have on the signal matrix, adjust the magnitude of the effect that is achieved due to processing the signal matrix based on the kernel matrix.

12. The apparatus of claim 11, wherein the logic that is operable to adjust the magnitude of the effect that is achieved due to processing the signal matrix based on the kernel matrix is operable to:

generate a modified kernel matrix for the signal matrix; and

apply the modified kernel matrix to the signal matrix to determine a new value for a first entry for the signal matrix.

13. The apparatus of claim 11, wherein the logic that is operable to adjust the magnitude of the effect that is achieved due to processing the signal matrix based on the kernel matrix is operable to:

apply the kernel matrix to the signal matrix to generate an intermediate pixel value; and

determine a weighted average of the intermediate pixel value and a first entry of the signal matrix.

14. The apparatus of claim 11, wherein the effect of applying the kernel matrix enhances image contrast in the signal.

15. The apparatus of claim 11, wherein the effect of applying the kernel matrix performs de-noising on the image in the signal.

16. The apparatus of claim 11, wherein the magnitude of the effect that the kernel matrix is intended to have on the signal matrix is based on a signal-to-noise ratio associated with the signal.

17. The apparatus of claim 16, wherein the magnitude of the effect that the kernel matrix is intended to have on the signal matrix is further based on properties of a lens that is used to capture the signal described by the signal matrices.

18. The apparatus of claim 17, wherein the magnitude of the effect that the kernel matrix is intended to have on the signal matrix is further based on characteristics of entries in the signal matrix.

19. The apparatus of claim 16, wherein the magnitude of the effect that the kernel matrix is intended to have on the signal matrix is further based on characteristics of entries in the signal matrix.

20. The apparatus of claim 11, wherein the magnitude of the effect that the kernel matrix is intended to have on the signal matrix is based on properties of a lens that is used to capture the signal described by the signal matrices.

21. The apparatus of claim 11, wherein the magnitude of the effect that the kernel matrix is intended to have on the signal matrix is based on characteristics of entries in the signal matrix.

22. The apparatus of claim 11, wherein the logic is further operable to determine the magnitude of the effect that the kernel matrix is intended to have on the signal matrix based on characteristics of entries in the signal matrix.

23. The apparatus of claim 22, wherein the logic is further operable to determine the metric is based on differences between values of the entries in the signal matrix.

24. The apparatus of claim 23, wherein the logic is further operable to determine the metric based on a standard deviation between the entries in the signal matrix.

25. The apparatus of claim 22, wherein the logic is further operable to determine the metric based on a histogram of the entries values in the signal matrix.

26. The apparatus of claim 22, wherein the logic is further operable to determine the metric by applying an edge detection mask to the signal matrix.

27. The apparatus of claim 22, wherein the logic is further operable to determine the metric based on entropy of values in the signal matrix.

28. The apparatus of claim 11, wherein the logic is further operable to, prior to processing the signal matrix based on the kernel matrix, modify the kernel matrix based on a motion blur characteristic of the image data.

29. An apparatus comprising:

a lens;

an image capture device that is optically coupled to the lens, wherein the image capture device captures image data from the lens; and

logic that is configured to simultaneously process the image data to correct for both optical aberration due to the lens and for motion blur.

30. The apparatus of claim 29, wherein to correct for the optical aberration the logic is configured to adjust, based on a signal-to-noise ratio of the image, an amount of sharpening to the image data.

31. The apparatus of claim 29, wherein to correct for the optical aberration the logic is configured to adjust, based on a lens shading profile of the lens, an amount of sharpening to the image data.

32. The apparatus of claim 29, wherein to correct for the optical aberration the logic is configured to adjust, based on local characteristics of the image data, an amount of sharpening to the image data.

33. The apparatus of claim 29, wherein the logic that is configured to process the image data is configured to correct for the motion blur based on an estimated aberration point spread function that is due to motion of the apparatus.

34. The apparatus of claim 29, wherein the logic that is configured to process the image data is configured to correct for the motion blur based on an estimated image shift detectable between a plurality of frames of the image data.