USING HIGHER ORDER STATISTICS TO ESTIMATE PIXEL VALUES IN DIGITAL IMAGE PROCESSING TO IMPROVE ACCURACY AND COMPUTATION EFFICIENCY

• US 20130202199A1
• Filed: 08/09/2011
• Published: 08/08/2013
• Est. Priority Date: 08/09/2010
• Status: Active Grant

• Current Assignee: Board of Regents of the University of Texas System (University of Texas System)
• Original Assignee: Board of Regents of the University of Texas System (University of Texas System)
• Inventors: Geisler, Wilson S., Perry, Jeffrey S.
• Granted Patent: US 9,064,190 B2
• Time in Patent Office: Days
• US Class Current: 382/159
• CPC Class Codes: G06T 3/4007 based on interpolation, e.g. bilinear interpolation image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403
• CPC Class Codes: H04N 1/387 Composing, repositioning or otherwise geometrically modifying originals
• CPC Class Codes: H04N 1/46 Colour picture communication systems
• CPC Class Codes: H04N 1/60 Colour correction or control H04N1/54 takes precedence

Abstract

A method, system and computer program product for improving accuracy and computation efficiency in interpolation, upsampling and color channel estimation. A Bayesian estimator used to estimate the value of a pixel in an image is constructed using measurements of high-order (e.g., 3rd, 4th, 5th) statics for nearby points in natural images. These measurements reveal highly systematic statistical regularities that were ignored from the prior algorithms due to their restrictive measurements and assumptions. As a result, the accuracy in interpolation, upsampling and color channel prediction is improved. Furthermore, the process for constructing a Bayesian estimator is simpler and more direct by storing in a table the mean value of the pixel value to be estimated for each combination of values of nearby points in training samples. As a result of having a simpler and more direct approach than existing methods, the computational efficiency is improved.

Claims

1. A method of determining an optimum candidate pixel value, the method comprising:

i. identifying a first plurality of candidate pixels in a training image wherein each of said plurality of candidate pixels is proximate to a first identical neighbor pixel group;

ii. for each of said first plurality of candidate pixels, determining a candidate pixel value thereby obtaining a first plurality of candidate pixel values;

iii. determining a plurality of statistical moments for said first plurality of candidate pixel values; and

iv. storing in memory a table including at least one of;

a first optimum candidate pixel value corresponding to said first identical neighbor pixel group, said first optimum candidate pixel being determined based on said plurality of statistical moments, and

information indicative of said plurality of statistical moments.

2. The method of claim 1, wherein:

step i further comprises identifying a plurality of additional candidate pixels in a plurality of additional training images;

step ii further comprises, for each of said additional candidate pixels, determining a candidate pixel value thereby obtaining an additional plurality of candidate pixel values; and

step iii further comprises determining said plurality of statistical moments corresponding to said first plurality of candidate pixel values and said additional plurality of candidate pixel values.

3. The method of claim 2, wherein, said plurality of additional training images are added in successive increments and a total number of said additional training images is determined based upon said determined statistical moments not changing beyond a threshold amount for two successive increments.

4. The method of claim 1, further comprising repeating each of steps i to iv for a second plurality of identical neighbor pixel groups thereby obtaining a second optimum candidate pixel value for each of said second plurality of identical neighbor pixel groups.

5. The method of claim 2, wherein, said plurality of additional training images are added in successive increments and a total number of said additional training images is determined based upon obtaining an optimum candidate pixel values for a threshold percentage of all possible unique identical neighbor pixel groups.

6. The method of claim 4, further comprising storing each of said optimum candidate pixel values in a table.

7. The method of claim 2, further comprising storing at least one of said statistical moments corresponding to each of said optimum candidate pixel values in said table.

8. A computer program product embodied in a computer readable storage medium for determining an optimum pixel value, the computer program product comprising programming instructions for:

i. identifying a first plurality of candidate pixels in a training image wherein each of said plurality of candidate pixels is proximate to a first identical neighbor pixel group;

ii. for each of said first plurality of candidate pixels, determining a candidate pixel value thereby obtaining a first plurality of candidate pixel values;

iii. determining a plurality of statistical moments for said first plurality of candidate pixel values; and

iv. storing in memory a table including at least one of;

a first optimum candidate pixel value corresponding to said first identical neighbor pixel group, said first optimum candidate pixel being determined based on said plurality of statistical moments, and

information indicative of said plurality of statistical moments.

9. A system for determining an optimum pixel value, the system comprising:

a memory unit; and

a processor coupled to said memory unit, wherein said processor is configured to;

i. identify a first plurality of candidate pixels in a training image wherein each of said plurality of candidate pixels is proximate to a first identical neighbor pixel group;

ii. for each of said first plurality of candidate pixels, determine a candidate pixel value thereby obtaining a first plurality of candidate pixel values;

iii. determine a plurality of statistical moments for said first plurality of candidate pixel values; and

iv. store in said memory unit a table including at least one of;

a first optimum candidate pixel value corresponding to said first identical neighbor pixel group, said first optimum candidate pixel being determined based on said plurality of statistical moments, and

information indicative of said plurality of statistical moments.

10. A method of estimating a missing pixel value, the method comprising:

identifying a missing pixel in an image, wherein said missing pixel is proximate to a first neighbor pixel group in said image; and

identifying, in a table of optimum pixel values stored in memory, an optimum pixel value for said missing pixel, said optimum pixel value being stored in said table in reference to said first neighbor pixel group, said optimum pixel value being previously determined based on a plurality of statistical moments for a first plurality of candidate pixel values located proximate to said first neighbor pixel group in at least one training image.

11. A computer program product embodied in a computer readable storage medium for estimating a missing pixel value, the computer program product comprising programming instructions for:

identifying a missing pixel in an image, wherein said missing pixel is proximate to a first neighbor pixel group in said image; and

identifying, in a table of optimum pixel values stored in memory, an optimum pixel value for said missing pixel, said optimum pixel value being stored in said table in reference to said first neighbor pixel group, said optimum pixel value being previously determined based on a plurality of statistical moments for a first plurality of candidate pixel values located proximate to said first neighbor pixel group in at least one training image.

12. A system for determining a missing pixel value, the system comprising:

a memory unit; and

a processor coupled to said memory unit, wherein said processor is configured to;

identify a missing pixel in an image, wherein said missing pixel is proximate to a first neighbor pixel group in said image; and

identify, in a table of optimum pixel values stored in said memory unit, an optimum pixel value for said missing pixel, said optimum pixel value being stored in said table in reference to said first neighbor pixel group, said optimum pixel value being previously determined based on a plurality of statistical moments for a first plurality of candidate pixel values located proximate to said first neighbor pixel group in at least one training image.

13. A method of determining an optimum high resolution candidate pixel value, the method comprising:

i. identifying a plurality of candidate pixels in a high resolution training image wherein each of said plurality of candidate pixels is located in a respective portion of said high resolution training image that corresponds to a plurality of respective portions of a low resolution training image, each of said plurality of portions of said low resolution training image comprising a first identical neighbor pixel group;

ii. for each of said plurality of candidate pixels, determining a candidate pixel value from said high resolution image thereby obtaining a plurality of candidate pixel values;

iii. determining a plurality of statistical moments for said plurality of candidate pixel values; and

iv. storing in memory a table including at least one of;

a first optimum candidate pixel value corresponding to said first identical neighbor pixel group, said first optimum candidate pixel being determined based on said plurality of statistical moments, and

information indicative of said plurality of statistical moments.

14. A computer program product embodied in a computer readable storage medium for determining an optimum high resolution candidate pixel value, the computer program product comprising programming instructions for:

i. identifying a plurality of candidate pixels in a high resolution training image wherein each of said plurality of candidate pixels is located in a respective portion of said high resolution training image that corresponds to a plurality of respective portions of a low resolution training image, each of said plurality of portions of said low resolution training image comprising a first identical neighbor pixel group;

ii. for each of said plurality of candidate pixels, determining a candidate pixel value from said high resolution image thereby obtaining a plurality of candidate pixel values;

iii. determining a plurality of statistical moments for said plurality of candidate pixel values; and

iv. storing in memory a table including at least one of;

a first optimum candidate pixel value corresponding to said first identical neighbor pixel group, said first optimum candidate pixel being determined based on said plurality of statistical moments, and

information indicative of said plurality of statistical moments.

15. A system for determining an optimum high resolution candidate pixel value, the system comprising:

a memory unit; and

a processor coupled to said memory unit, wherein said processor is configured to;

i. identify a plurality of candidate pixels in a high resolution training image wherein each of said plurality of candidate pixels is located in a respective portion of said high resolution training image that corresponds to a plurality of respective portions of a low resolution training image, each of said plurality of portions of said low resolution training image comprising a first identical neighbor pixel group;

ii. for each of said plurality of candidate pixels, determine a candidate pixel value from said high resolution image thereby obtaining a plurality of candidate pixel values;

iii. determine a plurality of statistical moments for said plurality of candidate pixel values; and

iv. store in said memory unit a table including at least one of;

a first optimum candidate pixel value corresponding to said first identical neighbor pixel group, said first optimum candidate pixel being determined based on said plurality of statistical moments, and

information indicative of said plurality of statistical moments.

16. A method of up-sampling an image, the method comprising:

identifying a missing pixel in a desired up-sampled image;

identifying a first portion of a low resolution image, said low resolution image being a lower resolution version of said desired up-sampled image, said first portion of said low resolution version comprising a first neighbor pixel group, said first pixel being located in a respective portion of said desired up-sampled image corresponding to said first portion of said low resolution image; and

identifying, in a table of optimum pixel values stored in memory, an optimum up-sampled pixel value for said missing pixel, said optimum pixel value being stored in said table in reference to said first neighbor pixel group, said optimum pixel value being previously determined based on a plurality of statistical moments for a first plurality of candidate pixel values located proximate to said first neighbor pixel group in at least one training image.

17. A computer program product embodied in a computer readable storage medium for upsampling an image, the computer program product comprising programming instructions for:

identifying a missing pixel in a desired up-sampled image;

identifying a first portion of a low resolution image, said low resolution image being a lower resolution version of said desired up-sampled image, said first portion of said low resolution version comprising a first neighbor pixel group, said first pixel being located in a respective portion of said desired up-sampled image corresponding to said first portion of said low resolution image; and

identifying, in a table of optimum pixel values stored in memory, an optimum up-sampled pixel value for said missing pixel, said optimum pixel value being stored in said table in reference to said first neighbor pixel group, said optimum pixel value being previously determined based on a plurality of statistical moments for a first plurality of candidate pixel values located proximate to said first neighbor pixel group in at least one training image.

18. A system for upsampling an image, the system comprising:

a memory unit; and

a processor coupled to said memory unit, wherein said processor is configured to;

identify a missing pixel in a desired up-sampled image;

identify a first portion of a low resolution image, said low resolution image being a lower resolution version of said desired up-sampled image, said first portion of said low resolution version comprising a first neighbor pixel group, said first pixel being located in a respective portion of said desired up-sampled image corresponding to said first portion of said low resolution image; and

identify, in a table of optimum pixel values stored in said memory unit, an optimum up-sampled pixel value for said missing pixel, said optimum pixel value being stored in said table in reference to said first neighbor pixel group, said optimum pixel value being previously determined based on a plurality of statistical moments for a first plurality of candidate pixel values located proximate to said first neighbor pixel group in at least one training image.

19. A method of determining an optimum color pixel value for a missing color channel, the method comprising:

i. identifying a plurality of candidate pixels in a training image wherein each of said plurality of candidate pixels comprises an identical first color channel pixel value and an identical second color channel pixel value, wherein said training image comprises a plurality of pixels, each of said pixels comprising a first color channel pixel value, a second color channel pixel value and a third color channel pixel value;

ii. for each of said plurality of candidate pixels, determining a third color channel candidate pixel value, thereby obtaining a plurality of third color channel candidate pixel values;

iii. determining a plurality of statistical moments for said plurality of third color channel candidate pixel values; and

iv. storing in memory a table including at least one of;

an optimum third color channel pixel value corresponding to said identical first color channel pixel value and said identical second color channel pixel value, said optimum third color channel pixel value being determined based on said plurality of statistical moments, and

information indicative of said plurality of statistical moments.

20. A computer program product embodied in a computer readable storage medium for determining an optimum color pixel value for a missing color channel, the computer program product comprising programming instructions for:

i. identifying a plurality of candidate pixels in a training image wherein each of said plurality of candidate pixels comprises an identical first color channel pixel value and an identical second color channel pixel value, wherein said training image comprises a plurality of pixels, each of said pixels comprising a first color channel pixel value, a second color channel pixel value and a third color channel pixel value;

ii. for each of said plurality of candidate pixels, determining a third color channel candidate pixel value, thereby obtaining a plurality of third color channel candidate pixel values;

iii. determining a plurality of statistical moments for said plurality of third color channel candidate pixel values; and

iv. storing in memory a table including at least one of;

an optimum third color channel pixel value corresponding to said identical first color channel pixel value and said identical second color channel pixel value, said optimum third color channel pixel value being determined based on said plurality of statistical moments, and

information indicative of said plurality of statistical moments.

21. A system for determining an optimum color pixel value for a missing color channel, the system comprising:

a memory unit; and

a processor coupled to said memory unit, wherein said processor is configured to;

i. identify a plurality of candidate pixels in a training image wherein each of said plurality of candidate pixels comprises an identical first color channel pixel value and an identical second color channel pixel value, wherein said training image comprises a plurality of pixels, each of said pixels comprising a first color channel pixel value, a second color channel pixel value and a third color channel pixel value;

ii. for each of said plurality of candidate pixels, determine a third color channel candidate pixel value, thereby obtaining a plurality of third color channel candidate pixel values;

iii. determine a plurality of statistical moments for said plurality of third color channel candidate pixel values; and

iv. store in the memory unit a table including at least one of;

an optimum third color channel pixel value corresponding to said identical first color channel pixel value and said identical second color channel pixel value, said optimum third color channel pixel value being determined based on said plurality of statistical moments, and

information indicative of said plurality of statistical moments.

22. A method of determining a color pixel value for a missing color channel, the method comprising:

identifying a first pixel in an image, said first pixel comprising a first color channel value and a second color channel value; and

identifying, in a table of optimum third color channel pixel values stored in memory, an optimum third color channel pixel value for said first pixel, said third color being a different color than said first color or said second color, said optimum third color channel pixel value being stored in said table in association with said first color channel value and said second color channel value of said first pixel, said optimum pixel value being previously determined based on a plurality of statistical moments for a first plurality of candidate third color channel pixel values of at least one training image, each of said candidate third color channel pixel values being located at pixels comprising said first color channel value and said second color channel value of said first pixel.

23. A computer program product embodied in a computer readable storage medium for determining a color pixel value for a missing color channel, the computer program product comprising programming instructions for:

identifying a first pixel in an image, said first pixel comprising a first color channel value and a second color channel value; and

identifying, in a table of optimum third color channel pixel values stored in memory, an optimum third color channel pixel value for said first pixel, said third color being a different color than said first color or said second color, said optimum third color channel pixel value being stored in said table in association with said first color channel value and said second color channel value of said first pixel, said optimum pixel value being previously determined based on a plurality of statistical moments for a first plurality of candidate third color channel pixel values of at least one training image, each of said candidate third color channel pixel values being located at pixels comprising said first color channel value and said second color channel value of said first pixel.

24. A system for determining a color pixel value for a missing color channel, the system comprising:

a memory unit; and

a processor coupled to said memory unit, wherein said processor is configured to;

identify a first pixel in an image, said first pixel comprising a first color channel value and a second color channel value; and

identify, in a table of optimum third color channel pixel values stored in the memory unit, an optimum third color channel pixel value for said first pixel, said third color being a different color than said first color or said second color, said optimum third color channel pixel value being stored in said table in association with said first color channel value and said second color channel value of said first pixel, said optimum pixel value being previously determined based on a plurality of statistical moments for a first plurality of candidate third color channel pixel values of at least one training image, each of said candidate third color channel pixel values being located at pixels comprising said first color channel value and said second color channel value of said first pixel.