Encoding and decoding system of motion image containing arbitrary object

• US 6,038,258 A
• Filed: 05/29/1997
• Issued: 03/14/2000
• Est. Priority Date: 05/29/1996
• Status: Expired due to Term

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Jeon, Byeung-Woo, Chun, Kang-Wook
• Primary Examiner(s): Britton, Howard
• Assistant Examiner(s): VO, TUNG T
• Application Number:
  US08/864,992
• Time in Patent Office: 1,020 Days
• Field of Search: 348/24, 348/25, 348/26, 348/413, 348/415, 348/416, 348/417, 348/699, 348/700, 348/845, 348/418, 345/420, 341/50, 358/183
• US Class Current: 375/240.12
• CPC Class Codes: H04N 19/20 using video object coding
• CPC Class Codes: H04N 19/537 Motion estimation other than block-based
• CPC Class Codes: H04N 19/54 using feature points or meshes
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

A motion image encoding apparatus for a current image containing an object uses mesh representation for encoding a motion image. The encoding apparatus includes an object extractor for extracting the object contained in the current image from received current image data and outputting object contour data indicating a contour of the extracted object. A predictive encoder performs a predictive encoding operation using the current image data, prestored data of a reference image, and control points of meshes which divide one of the current image and the reference image, and generates motion information relating to the control points, differential data between the current image and the reference image, and predictive image data. An object difference generator selects differential data in an object region among the differential data supplied from the predictive encoder to encode the selected differential data based on the object contour data output from the object extractor, and generates the selected differential data and the encoded differential data. An adder receives and adds the predictive image data output from the predictive encoder and the differential data selected in the object difference generator, and updates the reference image data prestored in the predictive encoder using the image data obtained by the addition result.

Claims

1. A motion image encoding apparatus for encoding a current image containing an object, the motion image encoding apparatus comprising:

object extraction means for extracting the object contained in the current image from received current image data and outputting object contour data indicating a contour of the extracted object;

predictive encoding means for performing a predictive encoding operation using the current image data, prestored data of a reference image, and control points of meshes which divide one of the current image and the reference image, and generating motion information relating to the control points, differential data between the current image and the reference image, and predictive image data;

object difference generator means for selecting differential data in an object region among the differential data supplied from the predictive encoding means to encode the selected differential data based on the object contour data output from the object extraction means, and outputting the selected differential data and the encoded differential data; and

adder means for receiving and adding the predictive image data output from the predictive encoding means and the selected differential data output from the object difference generator means to produce image data of an addition result, and updating the reference image data prestored in the predictive encoding means using the image data of the addition result.

2. The motion image encoding apparatus according to claim 1, wherein said predictive encoding means performs predictive encoding using spatial transformation to generate the predictive image data.

3. The motion image encoding apparatus according to claim 1, wherein said predictive encoding means comprises:

a memory for storing reference image data;

a mesh generator for determining regular meshes having vertices located in the object region of the current image as control points among the meshes for dividing the entire image into a predetermined size, based on the object contour data from said object extraction means and outputting mesh data representing determined regular meshes;

a motion estimation and compensation portion for generating the motion information according to the motion estimation with respect to the reference image of the control points, and the predictive image data whose motion of the control points is compensated, using the reference image data stored in said memory and the mesh data output from said mesh generator; and

a differential data generator for generating differential data between the current image data and the predictive image data.

4. The motion image encoding apparatus according to claim 3, wherein said mesh generator comprises:

a block former for outputting image block data representing a plurality of blocks each having a predetermined size obtained by dividing the current image with respect to the received current image data;

a block selector for selecting the image block data containing the image data of the object region determined by the object contour data among the image block data output from said block former, and outputting the selected image block data;

an object image composer for composing an object image by merging the image blocks of the selected image block data; and

a mesh composer for determining vertices contained in the object image constituted by the object image composer among vertices of a plurality of grids which divide the entire image in size, as control points, and determining the grids having the control points as rectangular meshes.

5. The motion image encoding apparatus according to claim 4, wherein said mesh composer outputs the mesh data representing regular triangular meshes obtained from the rectangular meshes.

6. The motion image encoding apparatus according to claim 5, wherein said regular triangular mesh is obtained by dividing the mesh at a diagonal direction of 45°

or -45°

based on data similarity between the control points of each regular rectangular mesh.

7. The motion image encoding apparatus according to claim 1, wherein said predictive encoding means comprises:

a memory for storing reference image data;

a mesh generator for determining irregular meshes having vertices located in the reference image as control points using the reference image data stored in said memory and outputting mesh data representing the determined irregular meshes;

a motion estimation and compensation portion for generating the motion information according to the motion estimation with respect to the current image of the control points, and the predictive image data whose motion of the control points is compensated, using the current image data and the mesh data output from said mesh generator; and

a differential data generator for generating differential data between the reference image data and the predictive image data.

8. The motion image encoding apparatus according to claim 7, wherein said mesh generator generates the mesh data representing the irregular meshes generated by irregular triangular mesh representation based on adaptive control removal.

9. The motion image encoding apparatus according to claim 1, wherein said object difference generator means performs orthogonal transform coding with respect to the selected differential data.

10. The motion image encoding apparatus according to claim 1, wherein said object difference generator means comprises:

a differential data encoder for selecting the differential data of the object region determined by the object contour data among the differential data output from said predictive encoding means, and encoding the selected differential data and the object contour data, to output the encoded differential data and the encoded object contour data; and

a differential data decoder for decoding the encoded differential data output from said differential encoder and outputting the selected differential data.

11. A motion image decoding apparatus for decoding an output of a motion image encoding apparatus, the motion image decoding apparatus comprising:

object contour restoring means for decoding received encoded object contour data and restoring object contour data indicating a contour of an object contained in a current image;

differential data decoding means for decoding received encoded differential data and outputting differential data;

motion compensation decoding means for performing a motion compensation operation using data of the object contour, prestored data of a reference image, control points of meshes which divide one of the object contour and the reference image, and motion information relating to the control points; and

adder means for adding the predictive image data selected by the object contour data and the differential data to produce image data of an addition result, and updating the reference image data prestored in the motion compensation decoding means using the image data of the addition result.

12. The motion image decoding apparatus according to claim 11, wherein said differential data decoding means performs orthogonal transform decoding with respect to the encoded differential data.

13. The motion image decoding apparatus according to claim 11, wherein said motion compensation decoding means performs motion compensation using spatial transformation to generate the predictive image data.

14. The motion image decoding apparatus according to claim 11, wherein said motion compensation decoding means comprises:

a memory for storing reference image data;

a mesh generator for determining meshes having vertices located in the object region determined by the object contour data output from said object contour restoring means as control points among the meshes by dividing the entire image into a predetermined size, and outputting mesh data representing the determined meshes and the object contour data; and

a motion compensation portion for generating the predictive image data whose motion is compensated with respect to the reference image of the control points, using the received motion information, the reference image data stored in said memory and the mesh data.

15. The motion image decoding apparatus according to claim 14, wherein the mesh data output from said mesh generator contains the position information of the vertices of the determined meshes.

16. The motion image decoding apparatus according to claim 14, wherein said mesh generator outputs mesh data relating to the regular rectangular meshes.

17. The motion image decoding apparatus according to claim 11, wherein said motion image decoding means comprises:

a memory for storing reference image data;

a mesh generator for determining meshes for dividing the reference image of the object region determined by the object contour data among the reference image data stored in said memory and outputting mesh data representing determined meshes; and

a motion estimation and compensation portion for generating the predictive image data whose motion is compensated with respect to the reference image of the control points of the determined meshes, using the received motion information, the reference image data stored in the memory, and the mesh data.

18. The motion image decoding apparatus according to claim 17, wherein said mesh generator generates the mesh data representing irregular meshes generated by irregular triangular mesh representation based on adaptive control point removal with respect to the reference image of the object region.

19. An object oriented image encoding apparatus for performing a motion prediction and compensation in a unit of an object image, the object-oriented image encoding apparatus comprising:

object contour information extraction means for extracting information representing a contour of an object image and outputting the extracted contour information;

mesh production means for representing the object image as a triangular or a rectangular mesh;

reference image input means for receiving a reference image to calculate a motion vector on a node point of the mesh in the object image represented as the mesh;

motion vector estimation means for receiving said reference image and the object image represented as the mesh and calculating the motion vector on the node point of the mesh;

motion compensation means for performing spatial transformation of a unit of the mesh in the object represented as the mesh using said motion vector on the node point of the mesh and producing a prediction data with respect to said reference image; and

transmission means for transmitting said contour information, mesh information and said motion vector on the node point of the mesh.

20. An object-oriented image encoding method for performing a motion prediction and compensation in a unit of an object image, the object-oriented image encoding method comprising the steps of:

(a) extracting information representing a contour of an object image and outputting the extracted contour information;

(b) representing the object image as a triangular or a rectangular mesh;

(c) receiving a reference image and the object image represented as the mesh and calculating a motion vector on a node point of the mesh;

(d) performing spatial transformation of a unit of the mesh in the object image represented as the mesh using said motion vector on the node point of the mesh and producing a prediction data with respect to said reference image; and

(e) transmitting said contour information, mesh information and said motion vector on the node point of the mesh.

21. An image decoding apparatus for performing a motion compensation in a unit of an object image, the image decoding apparatus comprising:

information input means for receiving contour information of an object image and a motion vector on a node point of a mesh;

object image contour restoration means for restoring a contour of the object image using said contour information of the object image;

mesh production means for restoring a mesh structure for the object image using said contour information of said object image; and

motion compensation means for performing spatial transformation as a unit of the mesh using a reference image, said motion vector on the node point of the mesh and the mesh structure for said restored object, and producing a prediction object image.

22. An image decoding method for performing a motion compensation in a unit of an object image, the image decoding method comprising the steps of:

(a) receiving contour information of an object image and a motion vector on a node point of a mesh;

(b) restoring a contour of the object image using said contour information of said object image;

(c) producing a mesh for restoring a mesh structure for the object image using said contour information of said object image;

(d) receiving a reference image for a motion compensation to the object image represented as said mesh; and

(e) performing spatial transformation as a unit of the mesh using a reference image, the motion vector on the node point of said mesh and the mesh structure for said restored object image, and producing a prediction object image.