Method and apparatus for encoding/decoding image using image residue prediction

• US 7,469,069 B2
• Filed: 05/14/2004
• Issued: 12/23/2008
• Est. Priority Date: 05/16/2003
• Status: Expired due to Fees

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kim, Woo-shik, Kim, Hyun-mun, Cho, Dae-Sung
• Primary Examiner(s): Do; Anh Hong
• Application Number:
  US10/845,256
• Publication Number:
  US 20050281473A1
• Time in Patent Office: 1,684 Days
• Field of Search: 382/232, 382/233, 382/251, 382/236, 382/239, 375/240.16, 375/240.17, 348/402.1, 348/699, 358/426.13, 345/418, 345/419, 386/52, 386/68, 386/109
• US Class Current: 382/236
• CPC Class Codes: H04N 19/105 Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/593 involving spatial prediction techniques
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

An encoding method may comprise obtaining a residue corresponding to a difference between the original image and a prediction image with respect to each component of the original image, encoding the residue of a predetermined component of the original image, restoring the encoded residue of the predetermined component and transforming the restored residue into a predetermined value including the restored residue value, and encoding prediction residues corresponding to differences between each of residues of components of the original image, which are not encoded, and the residue, which is restored and transformed into the predetermined value. Since prediction residues are encoded and decoded using correlations between residues of image components and image components, data amount is highly reduced. Also, since there is no quality loss in a picture generated by directly encoding an R-G-B image without transforming it into a Y-Cr-Cb image, high quality image information is obtained.

Claims

1. An image encoding method using residue prediction of an image, which encodes an original image including at least more than two components, the method comprising:

obtaining a residue corresponding to a difference between the original image and a prediction image with respect to each component of the original image;

encoding the residue of a predetermined component of the original image;

restoring the encoded residue of the predetermined component and transforming the restored residue into a predetermined value including the restored residue value; and

encoding prediction residues corresponding to differences between each of residues of components of the original image, which are not encoded, and the residue, which is restored and transformed into the predetermined value.

2. The method of claim 1, wherein the residue is transformed into the predetermined value using correlations among components of the original image.

3. The method of claim 1, wherein the original image is one of an R-G-B image, a Y-Cb-Cr image, and an X-Y-Z image.

4. An image encoding method using residue prediction of an image, the method comprising:

obtaining a temporal residue corresponding to a difference between an original image and a prediction image with respect to R-G-B components of a color image by estimating a motion between a current frame and a previous frame;

encoding a residue of the G component;

restoring the encoded residue of the G component and transforming the restored residue of the G component into a predetermined value including the restored residue value of the G component; and

encoding prediction residues corresponding to differences between each of residues of the R-B components of the original image, which are not encoded, and the residue of the G component transformed into the predetermined value.

5. The method of claim 4, wherein the motion estimation is performed in units of predetermined sized blocks.

6. The method of claim 4, wherein the restored residue of the G component is transformed into the predetermined value using correlations among the R-G-B components.

7. The method of claim 4, wherein the restored residue of the G component is transformed into the predetermined value using a linear transformation by multiplying the restored residue value of the G component by a predetermined value and adding the multiplied value to an offset.

8. The method of claim 7, wherein the linear transformation differently linear transforms block sizes used when a motion is compensated in units of blocks from the previous frame.

9. The method of claim 4, wherein the obtaining the residue of the G color component and the obtaining and encoding the prediction residues of the R and B color components using the residue of the G color component comprises obtaining a residue of the R color component and obtaining and encoding prediction residues of the G and B color components using the residue of the R color component.

10. The method of claim 4, wherein the obtaining the residue of the G component and the obtaining and encoding the prediction residues of the R and B components using the residue of the G color component comprises obtaining a residue of the B component and obtaining and encoding prediction residues of the G and R components using the residue of the B component.

11. An image encoding method using residue prediction of an image, the method comprising:

obtaining a spatial residue corresponding to a difference between an original image and a prediction image with respect to R-G-B components of a color image by estimating a predicted direction from pixels spatially close to a pixel block of a current frame;

encoding the residue of the G component;

restoring the encoded residue of the G component and transforming the restored residue of the G component into a predetermined value including the restored residue value of the G component; and

encoding prediction residues corresponding to differences between each of residues of the R-B components of the original image, which are not encoded, and the residue of the G component transformed into the predetermined value.

12. The method of claim 11, wherein the restored residue of the G component is transformed into the predetermined value using correlations among the R, G, and B components.

13. The method of claim 11, wherein the restored residue of the G component is transformed into the predetermined value using a linear transformation by multiplying the restored residue value of the G component by a predetermined value and adding the multiplied value to an offset.

14. The method of claim 13, wherein the linear transformation differently linear transforms spatially predicted directions used when residues of R-G-B components are obtained.

15. The method of claim 11, wherein the obtaining the residue of the G component and the obtaining and encoding the prediction residues of the R and B components using the residue of the G component comprises obtaining a residue of the R component and obtaining and encoding prediction residues of the G and B components using the residue of the R component.

16. The method of claim 11, wherein the obtaining the residue of the G component and the obtaining and encoding the prediction residues of the R and B components using the residue of the G component comprises obtaining a residue of the B component and obtaining and encoding prediction residues of the G and R components using the residue of the B component.

17. An image encoding method using residue prediction of an image, which encodes an R-G-B color image, the method comprising:

(1) in an inter mode, obtaining a prediction image by estimating a motion in units of predetermined sized blocks between a previous frame and a current frame with respect to each color component and generating a temporal residue by subtracting the prediction image from an original image;

(2) in an intra mode, obtaining a prediction image by estimating a predicted direction from spatially adjacent pixels of a pixel block of a current frame of each color component and generating a spatial residue by subtracting the prediction image from an original image;

(3) encoding the residue of the G component and restoring the encoded residue of the G component;

(4) obtaining prediction residues of the R-B components using the residue of the C component restored in step (3) and the residues of the R-B components generated in step (1) or (2);

(5) encoding the prediction residue values of the R-B components obtained in step (4) and restoring the encoded prediction residue images of the R-B components; and

(6) obtaining restored residue values of the R-G-B components using the restored residue image of the G component and the restored prediction residue images of the R-B components and generating restored images of the R-G-B components by adding the restored residue values of the R-G-B components to the prediction image value.

18. The method of claim 17, wherein step (3) comprises:

performing a frequency transformation, a quantization, and a variable length encoding of the residue of the G component to compress the residue of the G component and obtaining a restored residue image of the G component by performing an inverse quantization and an inverse frequency transformation on the quantized value again, and

step (5) comprises;

performing a frequency transformation, a quantization, and a variable length encoding of the prediction residue values of the R-B components obtained in step (4) and obtaining restored prediction residue images of the R-B components by performing an inverse quantization and an inverse frequency transformation on the quantized values again.

19. The method of claim 17, wherein step (4) comprises:

performing a linear transformation by multiplying the restored residue value of the G component by a predetermined value and adding the multiplied value to an offset; and

obtaining prediction residues corresponding to differences between each of the residues of the R-B components of the original image, which are not encoded, and the linear transformed residue of the G component.

20. An image encoding apparatus using residue prediction of an image, which encodes an original image including at least more than two components, the apparatus comprising:

a residue generator, which generates a difference between the original image and a prediction image with respect to each component of the original image;

a residue encoder, which encodes a residue of a predetermined component of the original image generated by the residue generator and prediction residues generated by a prediction residue generator;

a residue transformer, which restores the encoded residue of the predetermined component and transforms the restored residue into a predetermined value including the restored residue value; and

the prediction residue generator, which generates differences between each of residues of components of the original image, which are not encoded, and the residue, which is restored and transformed into the predetermined value.

21. An image encoding apparatus using residue prediction of an image, which encodes an R-G-B color image, the apparatus comprising:

a motion prediction image generator, which estimates a motion between a current frame and a previous frame in units of predetermined sized blocks and outputs a corresponding block;

a residue generator, which generates a temporal residue corresponding to a difference between an image generated by the motion prediction image generator and the corresponding block of the current frame with respect to R-G-B components of the color image;

a compression encoder, which compresses and encodes a residue of the G component generated by the residue generator and prediction residues of the R-B components generated by a prediction residue generator;

an inverse compression decoder, which decodes the compressed and encoded data and generates a restored residue of the G component and restored prediction residues of the R-B components;

a residue transformer, which transforms the restored residue of the G component into a predetermined value including the restored residue value of the G component using correlations among the R-G-B components; and

a prediction residue generator, which generates differences between each of the residues of the R-B components generated by the residue generator and the residue of the G component transformed by the residue transformer.

22. An image encoding apparatus using residue prediction of an image, which encodes an R-G-B color image, the apparatus comprising:

a spatial prediction image generator, which estimates a prediction direction from pixels spatially close to a pixel block of a current frame and outputs a corresponding block;

a residue generator, which generates a spatial residue corresponding to a difference between an image generated by the spatial prediction image generator and the corresponding block of the current frame with respect to R-G-B components of the color image;

a compression encoder, which compresses and encodes a residue of the G component generated by the residue generator and prediction residues of the R-B components generated by a prediction residue generator;

an inverse compression decoder, which decodes the compressed and encoded data and generates a restored residue of the G component and restored prediction residues of the R-B components;

a residue transformer, which transforms the restored residue of the G component into a predetermined value including the restored residue value of the G component using correlations among the R-G-B components; and

a prediction residue generator, which generates differences between each of the residues of the R-B components generated by the residue generator and the residue of the G component transformed by the residue transformer.

23. An image encoding apparatus using residue prediction of an image, which encodes an R-G-B color image, the apparatus comprising:

a motion prediction image generator, which estimates a motion between a current frame and a previous frame in units of predetermined sized blocks and outputs a corresponding block;

a spatial prediction image generator, which estimates a prediction direction from pixels spatially close to a pixel block of a current frame and outputs a corresponding block;

a residue generator, which generates a temporal residue corresponding to a difference between an image generated by the motion prediction image generator and the corresponding block of the current frame in an inter mode, or generates a spatial residue corresponding to a difference between an image generated by the spatial prediction image generator and the corresponding block of the current frame in an intra mode;

a compression encoder, which compresses and encodes a residue of the G component generated by the residue generator and prediction residues of the R-B components generated by a prediction residue generator;

an inverse compression decoder, which decodes the compressed and encoded data and generates a restored residue of the G component and restored prediction residues of the R-B components;

a residue transformer, which performs a linear transformation on the restored residue of the G component generated by the inverse compression decoder;

a prediction residue generator, which generates differences between each of the residues of the R-B components generated by the residue generator and the value transformed by the residue transformer;

a residue compensation unit, which generates restored residues of the R-B components by adding the prediction residues of the R-B components generated by the inverse compression decoder to the value transformed by the residue transformer; and

an original image generator, which generates an original image by adding the residues of the R-B components generated by the residue compensation unit and the residue of the G component generated by the inverse compression decoder to the prediction image generated by the motion prediction image generator in an inter mode, or generates an original image by adding the residues of the R-B components generated by the residue compensation unit and the residue of the G component generated by the inverse compression decoder to the prediction image generated by the spatial prediction image generator in an intra mode.

24. The apparatus of claim 23, wherein the motion prediction image generator comprises:

a motion estimation units, which estimates a motion between a current frame and a previous frame in units of predetermined sized blocks; and

a motion prediction unit, which outputs a block corresponding to a motion vector generated by the motion estimation unit, and

the spatial prediction image generator comprises;

a spatial estimation unit, which estimates a prediction direction from pixels spatially close to a pixel block of the current frame; and

a spatial prediction unit, which outputs a block corresponding to the direction predicted by the spatial estimation unit.

25. The apparatus of claim 23, wherein the compression encoder comprises:

a frequency transformer, which compresses the residue of the G component generated by the residue generator and the prediction residues of the R-B components generated by the prediction residue generator;

a quantizer, which quantizes data compressed by the frequency transformer; and

an entropy coder, which performs entropy encoding of the quantized data, and

the inverse compression decoder comprises;

an inverse quantizer, which inverse-quantizes the quantized data; and

an inverse frequency transformer, which performs an inverse frequency transformation of the inverse quantized data and generates restored residue of the G component and restored prediction residues of the R-B components.

26. An image decoding method using residue prediction of an image, which decodes an encoded original image including at least more than two components, where a residue is defined as a difference between the original image and a prediction image, and a prediction residue is defined as a difference between a residue of each component of the original image and a value that an encoded residue of a predetermined component is restored and transformed into a predetermined value, the method comprising:

decoding a residue of a predetermined component of the encoded original image;

transforming the decoded residue into a predetermined value;

decoding prediction residues of the components, which are not decoded;

obtaining residues of the components, which are not decoded, by adding the value transformed into the predetermined value to the prediction residues; and

restoring the original image by adding a prediction image to the residue of each component.

27. The method of claim 26, wherein the decoded residue is transformed into the predetermined value using correlations among components of the original image.

28. The method of claim 26, wherein the original image is one of an R-G-B type image, a Y-Cb-Cr type image, and an X-Y-Z type image.

29. An image decoding method using residue prediction of an image, which decodes an encoded original image including R-G-B components, where a temporal residue is defined as a difference between an original image and a prediction image by estimating a motion between a current frame and a previous frame with respect to each of the R-G-B components, and a prediction residue is defined as a difference between a value that an encoded residue of a predetermined component is restored and transformed into a predetermined value and a residue of a component, which is not transformed, the method comprising:

decoding a temporal residue of the G component from the encoded image including R-G-B components;

transforming the decoded residue of the G component into a predetermined value;

decoding prediction residues of the R-B components, which are not decoded;

obtaining residues of the R-B components by adding the value transformed into the predetermined value to the prediction residues; and

restoring the original image by adding a prediction image to the residues of the R-G-B components.

30. The method of claim 29, wherein the motion estimation is performed in units of predetermined sized blocks.

31. The method of claim 29, wherein the restored residue of the G component is transformed into the predetermined value using correlations among the R-G-B components.

32. The method of claim 29, wherein the restored residue of the G component is transformed into the predetermined value using a linear transformation by multiplying the restored residue value of the G component by a predetermined value and adding the multiplied value to an offset.

33. The method of claim 32, wherein the linear transformation differently linear transforms block sizes used when a motion is compensated in units of blocks from the previous frame.

34. The method of claim 29, wherein the obtaining the residue of the G component and the obtaining and encoding the prediction residues of the R and B components using the residue of the G component comprises obtaining a residue of the R component and obtaining and encoding prediction residues of the G and B components using the residue of the R component.

35. The method of claim 29, wherein the obtaining the residue of the G component and the obtaining and encoding the prediction residues of the R and B components using the residue of the G component comprises obtaining a residue of the B component and obtaining and encoding prediction residues of the G and R components using the residue of the B component.

36. An image decoding method using residue prediction of an image, which decodes an encoded original image including R-G-B components, where a spatial residue is defined as a difference between an original image and a prediction image in which a prediction direction is estimated from pixels spatially close to a current frame with respect to the R-G-B components, and a prediction residue is defined as a difference between a value that an encoded spatial residue of a predetermined component is restored and transformed into a predetermined value and a residue of a component, which is not transformed, the method comprising:

decoding a residue of the G component;

transforming the decoded residue of the G component into a predetermined value including the decoded residue value of the G component;

decoding prediction residues of the R-B components, which are not decoded;

obtaining residues of the R-B components by adding the value transformed the residue of the G component into the predetermined value to the prediction residues; and

restoring the original image by adding a prediction image to the residues of the R-G-B components.

37. The method of claim 36, wherein the restored residue of the G component is transformed into the predetermined value using correlations among the R-G-B components.

38. The method of claim 36, wherein the restored residue of the G component is transformed into the predetermined value using a linear transformation by multiplying the restored residue value of the G component by a predetermined value and adding the multiplied value to an offset.

39. The method of claim 38, wherein the linear transformation differently linear transforms spatially predicted directions used when residues of R-G-B components are obtained.

40. The method of claim 36, wherein the decoding the residue of the G component and the decoding the prediction residues of the R and B components using the residue of the G component comprises decoding a residue of the R plane and decoding prediction residues of the G and B components using the residue of the R component.

41. The method of claim 36, wherein the decoding the residue of the G component and the decoding the prediction residues of the R and B components using the residue of the G component comprises decoding a residue of the B component and decoding prediction residues of the G and R components using the residue of the B component.

42. An image decoding method using residue prediction of an image, which decodes an R-G-B color image, where a residue is defined as a difference between the color image and a prediction image, and a prediction residue is defined as a difference between a residue of each component of the original image and a value that an encoded residue of a predetermined component is restored and transformed into a predetermined value, the method comprising:

(1) restoring a residue of a G component image from encoded image data;

(2) obtaining residue images of R-B components from encoded image data and generating restored residue images of R-B components by adding the residue images of R-B components to a value obtained by linearly transforming the restored residue value obtained in step (1);

(3) in an inter mode, restoring a color image of the R-G-B components by adding the residue values of the R-G-B components restored in steps (1) and (2) to a prediction image obtained by performing temporal motion compensation of color components of a previous frame; and

(4) in an intra mode, restoring a color image of the R-G-B components by adding the residue values of the R-G-B components restored in steps (1) and (2) to a prediction image obtained by estimating a predicted direction from spatially adjacent pixels.

43. The method of claim 42, wherein step (1) comprises:

obtaining a restored residue image of the G component by performing an inverse variable length decoding, an inverse quantization, and an inverse frequency transformation on the encoded data, and

step (2) comprises;

obtaining the residue images of the R-B components by performing an inverse variable length decoding, an inverse quantization, and an inverse frequency transformation of the encoded data and generating the restored residue images of the R-B components by adding the residue images of the R-B components to a value obtained by linearly transforming the restored residue value obtained in step (1).

44. An image decoding apparatus using residue prediction of an image, which decodes an encoded original image including at least more than two components, where a residue is defined as a difference between the original image and a prediction image, and a prediction residue is defined as a difference between a residue of each component of the original image and a value that an encoded residue of a predetermined component is restored and transformed into a predetermined value, the apparatus comprising:

an inverse compression decoder, which decodes the encoded image data and generates a restored residue image of a predetermined component of the image and restored prediction residues of components corresponding to residues, which are not restored;

a residue transformer, which transforms the restored residue of the component generated by the inverse compression decoder into a predetermined value including the restored residue value of the component;

a residue compensation unit, which generates restored residues by adding the prediction residues generated by the inverse compression decoder to the value transformed by the residue transformer; and

an original image generator, which generates an original image by adding a prediction image to each of the residues generated by the residue compensation unit and the residue of the predetermined component generated by the inverse compression decoder.

45. An image decoding apparatus using residue prediction of an image, which decodes an encoded original image including R-G-B components, where a temporal residue is defined as a difference between an original image and a prediction image by estimating a motion between a current frame and a previous frame with respect to the R-G-B components, and a prediction residue is defined as a difference between a value that an encoded residue of a predetermined component is restored and transformed into a predetermined value and a residue of a component, which is not transformed, the apparatus comprising:

a motion compensation unit, which generates a temporally motion compensated prediction image;

an inverse compression decoder, which decodes the encoded image data and generates a restored residue image of the G component and restored prediction residues of the R-B components;

a residue transformer, which performs a linear transformation on the restored residue of the G component generated by the inverse compression decoder;

a residue compensation unit, which generates restored residues of the R-B components by adding the prediction residues of the R-B components generated by the inverse compression decoder to the value transformed by the residue transformer; and

an original image generator, which generates an original image by adding a prediction image generated by the motion compensation unit to the residues of the R-B component generated by the residue compensation unit and the residue of the G component generated by inverse compression decoder.

46. An image decoding apparatus using residue prediction of an image, which decodes an encoded original image including R-G-B components, where a spatial residue is defined as a difference between an original image and a prediction image by estimating a prediction direction from pixels spatially close to a current frame with respect to the R-G-B components, and a prediction residue is defined as a difference between a value that an encoded spatial residue of a predetermined component is restored and transformed into a predetermined value and a residue of a component, which is not transformed, the apparatus comprising:

a spatial prediction unit, which generates a prediction image obtained by estimating a prediction direction from spatially adjacent pixels;

an inverse compression decoder, which decodes the encoded image data and generates a restored residue image of the G component and restored prediction residues of the R-B components;

a residue transformer, which performs a linear transformation on the restored residue of the G component generated by the inverse compression decoder;

a residue compensation unit, which generates restored residues of the R-B components by adding the prediction residues of the R-B components generated by the inverse compression decoder to the value transformed by the residue transformer; and

an original image generator, which generates an original image by adding a prediction image generated by the spatial prediction unit to the residues of the R-B components generated by the residue compensation unit and residue of the G component generated by inverse compression decoder.

47. An image decoding apparatus using residue prediction of an image, which decodes an encoded original image including at least more than two components, where a residue is defined as a difference between the original image and a prediction image, and a prediction residue is defined as a difference between a residue of each component of the original image and a value that an encoded residue of a predetermined component is restored and transformed into a predetermined value, the apparatus comprising:

a motion compensation unit, which generates a temporally motion compensated prediction image;

a spatial prediction unit, which generates a prediction image obtained by estimating a prediction direction from spatially adjacent pixels;

an inverse compression decoder, which decodes the encoded image data and generates a restored residue image of a G component and restored prediction residues of R-B components;

a residue transformer, which performs a linear transformation on the restored residue of the G component generated by the inverse compression decoder;

a residue compensation unit, which generates restored residues of R-B components by adding the prediction residues of R-B components generated by the inverse compression decoder to the value transformed by the residue transformer; and

an original image generator, which generates an original image by adding the residues of R-B components generated by the residue compensation unit and the residue of the G component generated by inverse compression decoder to a prediction image generated by the motion compensation unit in a inter mode or to a prediction image generated by the spatial prediction unit in an intra mode.

48. The apparatus of claim 47, wherein the inverse compression decoder comprises:

an entropy decoder, which performs entropy decoding of the encoded image data;

an inverse quantizer, which inverse-quantizes the entropy decoded data; and

an inverse frequency transformer, which generates the restored residue of the G component and the restored prediction residues of R-B components by performing an inverse frequency transformation on the inverse quantized data.

49. A computer readable medium having recorded thereon a computer readable program for performing an image encoding method using residue prediction of an image, which encodes an original image including at least more than two components, the method comprising

obtaining a residue corresponding to a difference between the original image and a prediction image with respect to each component of the original image;

encoding the residue of a predetermined component of the original image;

restoring the encoded residue of the predetermined component and transforming the restored residue into a predetermined value including the restored residue value; and

encoding prediction residues corresponding to differences between each of residues of components of the original image, which are not encoded, and the residue, which is restored and transformed into the predetermined value.

50. A computer readable medium having recorded thereon a computer readable program for performing an image decoding method using residue prediction of an image, which decodes an encoded original image including at least more than two components, where a residue is defined as a difference between the original image and a prediction image, and a prediction residue is defined as a difference between a residue of each component of the original image and a value that an encoded residue of a predetermined component is restored and transformed into a predetermined value, the method comprising:

decoding a residue of a predetermined component of the encoded original image;

transforming the decoded residue into a predetermined value;

decoding prediction residues of the components, which are not decoded;

obtaining residues of the components, which are not decoded, by adding the value transformed into the predetermined value to the prediction residues; and

restoring the original image by adding a prediction image to the residue of each component.