Adaptive Generation Of Irregular Spatial Sub-sampling For Images

• US 20090268978A1
• Filed: 04/29/2008
• Published: 10/29/2009
• Est. Priority Date: 04/29/2008
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Inventors: Paniconi, Marco
• Granted Patent: US 8,059,909 B2
• Time in Patent Office: Days
• US Class Current: 382/261
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: H04N 19/105 Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/593 involving spatial prediction techniques
• CPC Class Codes: H04N 19/94 Vector quantisation

Abstract

An adaptive filter is applied to samples in an initial pattern of samples. The samples in the initial pattern correspond to a subset of the image pixels which are to be predicted using the samples. A level value of each sample in the pattern is varied. The level value corresponds to the intensity/signal value of the sample, and may have a strong effect on the prediction power of the samples. A first prediction pattern (i.e., the prediction signal corresponding to the sampling pattern) is generated from the samples. A sample movement process is performed on a current sample in the first prediction pattern to change a current position of the current sample in the first prediction pattern. A local prediction of pixels influenced by the sample is updated if the sample'"'"'s position is changed. A sample removal process is performed on the current sample to remove the sample from the first pattern if a criteria is satisfied. A local prediction of pixels is updated if the current sample is removed. A second prediction pattern (i.e., the prediction signal corresponding to the new/second sampling pattern) is generated from the samples remaining in the first pattern. The second pattern is output.

Claims

1. A computerized method comprising:

applying an adaptive filter to samples in an initial pattern of samples, wherein the samples in the initial pattern correspond to pixels to be predicted;

varying a level value of each sample, wherein the level value corresponds to a predictive strength of the sample in relation to the pixels to be predicted;

generating a first prediction pattern from the samples;

performing a sample movement process on a current sample in the first prediction pattern to change a current position of the current sample in the first prediction pattern;

updating a local prediction of pixels influenced by the current sample if the current position of the current sample is changed;

performing a sample removal process on the current sample in the first prediction pattern to remove the current sample from the first prediction pattern if a criteria is satisfied;

updating a local prediction of pixels influenced by the current sample if the current sample is removed;

generating a second prediction pattern from the samples remaining in the first prediction pattern; and

outputting the second prediction pattern.

2. The computerized method of claim 1, wherein the sample movement process comprises:

computing a different sample position for the current sample;

computing a current prediction error corresponding to the current position;

computing a different prediction error corresponding to the different sample position; and

moving the current sample to a position with a lowest prediction error.

3. The computerized method of claim 2, wherein the updating a local prediction comprises:

computing an area of influence of the sample, the area of influence including pixels for which the sample is a factor in a prediction thereof;

updating a local prediction error for pixels in the area of influence of the current sample;

updating filter coefficients for the pixels in the area of influence, wherein the filter coefficients define a prediction relationship between the current sample and the pixels;

applying the adaptive filter to the pixels in the area of influence; and

applying the level variation to the current sample.

4. The computerized method of claim 1, wherein the sample removal process comprises:

determining, from a current prediction of the pixels in the area of influence, a current prediction error, the current prediction including the predictive weight of the current sample;

determining, from a test prediction of the pixels in the area of influence, a test prediction error, the test prediction excluding the predictive weight of the current sample;

removing the current sample if an absolute value of a difference between the current prediction error and the test prediction error is less than a sample removal threshold.

5. The computerized method of claim 4, wherein the removing the current sample further comprises:

determining a sample density, the determining excluding the presence of the current sample;

removing the current sample if the sample density is greater than a minimum sample density.

6. The computerized method of claim 4, the method further comprising:

computing an area of influence of the sample, the area of influence including pixels for which the sample is a factor in a prediction thereof;

updating, if the current sample is removed, a local prediction error for pixels in the area of influence of the current sample;

updating, if the current sample is removed, filter coefficients for the pixels in the area of influence, wherein the filter coefficients define a prediction relationship between the current sample and the pixels;

applying, if the current sample is removed, the adaptive filter to the pixels in the area of influence; and

applying the level variation to the current sample if the current sample remains in the first prediction pattern.

7. A computer-readable storage medium storing instructions which when executed by a data processing system cause the data processing system to perform operations that process data, the operations comprising:

applying an adaptive filter to samples in an initial pattern of samples, wherein the samples in the initial pattern correspond to pixels to be predicted;

varying a level value of each sample, wherein the level value corresponds to a predictive strength of the sample in relation to the pixels to be predicted;

generating a first prediction pattern from the samples;

performing a sample movement process on a current sample in the first prediction pattern to change a current position of the current sample in the first prediction pattern;

updating a local prediction of pixels influenced by the current sample if the current position of the current sample is changed;

performing a sample removal process on the current sample in the first prediction pattern to remove the current sample from the first prediction pattern if a criteria is satisfied;

updating a local prediction of pixels influenced by the current sample if the current sample is removed;

generating a second prediction pattern from the samples remaining in the first prediction pattern; and

outputting the second prediction pattern.

8. The computer-readable storage medium of claim 7, wherein the sample movement process comprises:

computing a different sample position for the current sample;

computing a current prediction error corresponding to the current position;

computing a different prediction error corresponding to the different sample position; and

moving the current sample to a position with a lowest prediction error.

9. The computer-readable storage medium of claim 8, wherein the updating a local prediction comprises:

computing an area of influence of the sample, the area of influence including pixels for which the sample is a factor in a prediction thereof;

updating a local prediction error for pixels in the area of influence of the current sample;

updating filter coefficients for the pixels in the area of influence, wherein the filter coefficients define a prediction relationship between the current sample and the pixels;

applying the adaptive filter to the pixels in the area of influence; and

applying the level variation to the current sample.

10. The computer-readable storage medium of claim 7, wherein the sample removal process comprises:

determining, from a current prediction of the pixels in the area of influence, a current prediction error, the current prediction including the predictive weight of the current sample;

determining, from a test prediction of the pixels in the area of influence, a test prediction error, the test prediction excluding the predictive weight of the current sample;

removing the current sample if an absolute value of a difference between the current prediction error and the test prediction error is less than a sample removal threshold.

11. The computer-readable storage medium of claim 10, wherein the removing the current sample further comprises:

determining a sample density, the determining excluding the presence of the current sample;

removing the current sample if the sample density is greater than a minimum sample density.

12. The computer-readable storage medium of claim 10, the operations further comprising:

computing an area of influence of the sample, the area of influence including pixels for which the sample is a factor in a prediction thereof;

updating, if the current sample is removed, a local prediction error for pixels in the area of influence of the current sample;

updating, if the current sample is removed, filter coefficients for the pixels in the area of influence, wherein the filter coefficients define a prediction relationship between the current sample and the pixels;

applying, if the current sample is removed, the adaptive filter to the pixels in the area of influence; and

applying the level variation to the current sample if the current sample remains in the first prediction pattern.

13. A system comprising:

a processor coupled to a bus;

a memory coupled to the processor through the bus; and

instructions executed by the processor from the memory to cause the processor

to apply an adaptive filter to samples in an initial pattern of samples, wherein the samples in the initial pattern correspond to pixels to be predicted,

vary a level value of each sample, wherein the level value corresponds to a predictive strength of the sample in relation to the pixels to be predicted,

generate a first prediction pattern from the samples,

perform a sample movement process on a current sample in the first prediction pattern to change a current position of the current sample in the first prediction pattern,

update a local prediction of pixels influenced by the current sample if the current position of the current sample is changed,

perform a sample removal process on the current sample in the first prediction pattern to remove the current sample from the first prediction pattern if a criteria is satisfied,

update a local prediction of pixels influenced by the current sample if the current sample is removed,

generate a second prediction pattern from the samples remaining in the first prediction pattern,

and output the second prediction pattern.

14. The system of claim 13, wherein the sample movement process comprises:

computing a different sample position for the current sample;

computing a current prediction error corresponding to the current position;

computing a different prediction error corresponding to the different sample position; and

moving the current sample to a position with a lowest prediction error.

15. The system of claim 14, wherein the updating a local prediction comprises:

computing an area of influence of the sample, the area of influence including pixels for which the sample is a factor in a prediction thereof;

updating a local prediction error for pixels in the area of influence of the current sample;

updating filter coefficients for the pixels in the area of influence, wherein the filter coefficients define a prediction relationship between the current sample and the pixels;

applying the adaptive filter to the pixels in the area of influence; and

applying the level variation to the current sample.

16. The system of claim 13, wherein the sample removal process comprises:

determining, from a current prediction of the pixels in the area of influence, a current prediction error, the current prediction including the predictive weight of the current sample;

determining, from a test prediction of the pixels in the area of influence, a test prediction error, the test prediction excluding the predictive weight of the current sample;

removing the current sample if an absolute value of a difference between the current prediction error and the test prediction error is less than a sample removal threshold.

17. The system of claim 16, wherein the removing the current sample further comprises:

determining a sample density, the determining excluding the presence of the current sample;

removing the current sample if the sample density is greater than a minimum sample density.

18. The system of claim 16, the instructions further comprising:

computing an area of influence of the sample, the area of influence including pixels for which the sample is a factor in a prediction thereof;

updating, if the current sample is removed, a local prediction error for pixels in the area of influence of the current sample;

updating, if the current sample is removed, filter coefficients for the pixels in the area of influence, wherein the filter coefficients define a prediction relationship between the current sample and the pixels;

applying, if the current sample is removed, the adaptive filter to the pixels in the area of influence; and

*applying the level variation to the current sample if the current sample remains in the first prediction pattern.

19. An apparatus comprising:

means for receiving image data, the image data comprising an initial pattern of samples;

means for applying an adaptive filter to samples in the initial pattern of samples, wherein the samples in the initial pattern correspond to pixels to be predicted;

means for varying a level value of each sample, wherein the level value corresponds to a predictive strength of the sample in relation to the pixels to be predicted;

means for generating a first prediction pattern from the samples;

means for performing a sample movement process on a current sample in the first prediction pattern to change a current position of the current sample in the first prediction pattern;

means for updating a local prediction of pixels influenced by the current sample if the current position of the current sample is changed;

means for performing a sample removal process on the current sample in the first prediction pattern to remove the current sample from the first prediction pattern if a criteria is satisfied;

means for updating a local prediction of pixels influenced by the current sample if the current sample is removed;

means for generating a second prediction pattern from the samples remaining in the first prediction pattern; and

means for outputting the second prediction pattern.