IMAGE PROCESSING DEVICE, IMAGE PROCESSING METHOD, AND PROGRAM

• US 20130216150A1
• Filed: 12/07/2011
• Published: 08/22/2013
• Est. Priority Date: 12/08/2010
• Status: Abandoned Application

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Kondo, Kenji
• Application Number:
  US13/881,595
• Publication Number:
  US 20130216150A1
• Time in Patent Office: Days
• US Class Current: 382/238
• CPC Class Codes: G06T 9/004 Predictors, e.g. intraframe, interframe coding
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/182 the unit being a pixel
• CPC Class Codes: H04N 19/43 Hardware specially adapted for motion estimation or compensation
• CPC Class Codes: H04N 19/523 with sub-pixel accuracy
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/82 involving filtering within a prediction loop

Abstract

The present invention relates to an image processing device, an image processing method, and a program that can reduce usage of memory bandwidth when motion compensation operations with fractional precision are performed in inter predictions. A reference image read unit reads predetermined reference pixels from a reference image in an inter prediction so that the number of the predetermined reference pixels corresponding to an outer pixel in a predicted image in the inter prediction is smaller than the number of the predetermined reference pixels corresponding to an inner pixel in the predicted image. Using the read predetermined reference pixels, a 2-tap FIR filter, a 4-tap FIR filter, or a 6-tap FIR filter calculates a pixel having a fractional position in the reference image as a pixel in the predicted image. The present technique can be applied to encoding devices that perform compression encoding by H.264/AVC, for example.

Claims

1. An image processing device comprising:

a pixel read unit configured to read predetermined pixels from a reference image in an inter prediction; and

an arithmetic operation unit configured to calculate a pixel having a fractional position in the reference image as a pixel in a predicted image in the inter prediction, by using the predetermined pixels read by the pixel read unit,

wherein the pixel read unit reads the predetermined pixels so that the number of the predetermined pixels corresponding to an outer pixel in the predicted image is smaller than the number of the predetermined pixels corresponding to an inner pixel in the predicted image.

2. The image processing device according to claim 1, wherein a position of the outer pixel in the reference image is a position outside a range of a size of the predicted image in a reference block formed with all the predetermined pixels to be used in calculating respective pixels in the predicted image.

3. The image processing device according to claim 2, wherein the pixel read unit reads the predetermined pixels so that pixels located outside the range of the size of the predicted pixel in the reference block among the predetermined pixels corresponding to the outer pixel are identical to pixels located outside the range of the size of the predicted image in the reference block among the predetermined pixels corresponding to the inner pixel.

4. The image processing device according to claim 1, wherein a position of the outer pixel in the reference image is a position located on a boundary between a range of a size of the predicted image in a reference block formed with all the predetermined pixels to be used in calculating respective pixels in the predicted image and a range located outside the range of the size of the predicted image.

5. The image processing device according to claim 1, wherein the pixel read unit reads the predetermined pixels so that the number of the predetermined pixels corresponding to an outermost pixel in the predicted image is two.

6. The image processing device according to claim 1, wherein the pixel read unit reads the predetermined pixels so that the number of the predetermined pixels corresponding to an outermost pixel in the predicted image is two, the number of the predetermined pixels corresponding to a second outermost pixel located on an inner side of the outermost pixel is four, and the number of the predetermined pixels corresponding to one of pixels that are located on an inner side of the second outermost pixel and include innermost pixels in the predicted image is six.

7. The image processing device according to claim 1, further comprising

a storage unit configured to store outer coefficients corresponding to an outer pixel in the predicted image and inner coefficients corresponding to an inner pixel in the predicted image,

wherein, when the predetermined pixels read by the pixel read unit correspond to the outer pixel in the predicted image, the arithmetic operation unit performs the calculation by using the predetermined pixels and the outer coefficients, and, when the predetermined pixels read by the pixel read unit correspond to the inner pixel in the predicted image, the arithmetic operation unit performs the calculation by using the predetermined pixels and the inner coefficients.

8. The image processing device according to claim 1, further comprising

a pixel generation unit configured to, when the number of the predetermined pixels read by the pixel read unit is smaller than a predetermined number, replicate an outer pixel among the predetermined pixels by a difference number between the number of the predetermined pixels and the predetermined number, and generate the predetermined number of pixels including a replicated pixel and the predetermined pixels,

wherein the arithmetic operation unit calculates a pixel in the predicted image by using the predetermined number of pixels generated by the pixel generation unit.

9. The image processing device according to claim 8, wherein the pixel read unit reads the predetermined pixels so that the number of the predetermined pixels corresponding to an outermost pixel in the predicted image is four, the number of the predetermined pixels corresponding to a second outermost pixel located on an inner side of the outermost pixel is five, and the number of the predetermined pixels corresponding to one of pixels that are located on an inner side of the second outermost pixel and include innermost pixels in the predicted image is six, and

the predetermined number is six.

10. The image processing device according to claim 1, wherein, when each pixel in the predicted image has a fractional position in only one of a horizontal direction and a vertical direction in the reference image, the pixel read unit reads the predetermined pixels so that the number of the predetermined pixels corresponding to an outer pixel in the predicted image is larger than when each pixel in the predicted image has a fractional position in both of the directions.

11. The image processing device according to claim 1, wherein, when a size of the predicted image is larger than a predetermined size, the pixel read unit reads the predetermined pixels so that the number of the predetermined pixels corresponding to an outer pixel in the predicted image is larger than when the size of the predicted image is equal to or smaller than the predetermined size.

12. The image processing device according to claim 1, wherein, when the inter prediction has a predicting direction of a unidirectional prediction, the pixel read unit reads the predetermined pixels so that the number of the predetermined pixels corresponding to an outer pixel in the predicted image is larger than when the inter prediction is a bidirectional prediction.

13. The image processing device according to claim 1, wherein the pixel read unit reads the predetermined pixels so that the number of the predetermined pixels that correspond to an outer pixel in the predicted image and are located on an outer side of the predicted pixel is smaller than the number of the predetermined pixels that correspond to an inner pixel in the predicted image and are located on an outer side of the predicted pixel.

14. The image processing device according to claim 13, further comprising

a pixel generation unit configured to, when the number of the predetermined pixels read by the pixel read unit is smaller than a predetermined number, replicate an outer pixel among the predetermined pixels by a difference number between the number of the predetermined pixels and the predetermined number, and generate the predetermined number of pixels including a replicated pixel and the predetermined pixels,

wherein the arithmetic operation unit calculates a pixel in the predicted image by using the predetermined number of pixels generated by the pixel generation unit.

15. The image processing device according to claim 1, wherein the pixel read unit reads the predetermined pixels so that the number of the predetermined pixels corresponding to an outer pixel in the predicted image is smaller than the number of the predetermined pixels corresponding to an inner pixel in the predicted image, and the number of the predetermined pixels aligned in one of a horizontal direction and a vertical direction is smaller than the number of the predetermined pixels aligned in the other one of the horizontal direction and the vertical direction.

16. The image processing device according to claim 1, further comprising

a difference calculation unit configured to calculate a difference between the predicted image and an encoding target image.

17. The image processing device according to claim 1, further comprising

an adding operation unit configured to add the predicted image to a decoding target image.

18. An image processing method to be performed by an image processing device, comprising:

a pixel read step of reading predetermined pixels from a reference image in an inter prediction; and

an arithmetic operation step of calculating a pixel having a fractional position in the reference image as a pixel in a predicted image in the inter prediction, by using the predetermined pixels read through the procedure of the pixel read step,

wherein the procedure of the pixel read step includes reading the predetermined pixels so that the number of the predetermined pixels corresponding to an outer pixel in the predicted image is smaller than the number of the predetermined pixels corresponding to an inner pixel in the predicted image.

19. A program for causing a computer to execute:

a pixel read step of reading predetermined pixels from a reference image in an inter prediction; and

an arithmetic operation step of calculating a pixel having a fractional position in the reference image as a pixel in a predicted image in the inter prediction, by using the predetermined pixels read through the procedure of the pixel read step,

wherein the procedure of the pixel read step includes reading the predetermined pixels so that the number of the predetermined pixels corresponding to an outer pixel in the predicted image is smaller than the number of the predetermined pixels corresponding to an inner pixel in the predicted image.