Transparent block skipping in object-based video coding systems

• US 5,748,789 A
• Filed: 10/31/1996
• Issued: 05/05/1998
• Est. Priority Date: 10/31/1996
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Lee, Ming-Chieh, Chen, Wei-ge
• Primary Examiner(s): Boudreau, Leo
• Assistant Examiner(s): Tadayon, Bijan
• Application Number:
  US08/741,949
• Time in Patent Office: 551 Days
• Field of Search: 382/108, 382/164, 382/165, 382/166, 382/170, 382/173, 382/181, 382/190, 382/195, 382/199, 382/203, 382/209, 382/224, 382/232, 382/233, 382/234, 382/235, 382/236, 382/238, 382/239, 382/240, 382/241, 382/242, 382/243, 382/244, 382/248, 382/251, 382/282, 382/308, 382/197, 395/133, 348/415, 348/14, 348/416, 348/402, 348/384, 348/403, 348/407, 348/420, 358/462, 358/261.3, 345/139
• US Class Current: 382/243
• CPC Class Codes: H04N 19/20 using video object coding
• CPC Class Codes: H04N 19/503 involving temporal prediction adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109

Abstract

A method implemented in an object-based video encoder or decoder uses shape information that describes the boundary of a group of pixels representing an object in a sequence of video frames to identify transparent blocks (e.g., macroblocks or blocks so that coding/decoding of these blocks can be skipped. In the object-based video coding method, encoders code shape separately from motion and texture, and shape information is available before the encoder/decoder codes/decodes texture and motion data. The encoder and decoder use this shape information to identify transparent macroblocks or blocks so that texture coding and possible motion coding can be skipped. This method for transparent block skipping reduces coding and decoding operations and reduces the number of bits needed to store a bitstream representing a compressed video sequence.

Claims

1. In an object-based video coding method, a method for reducing coding overhead comprising:

separately encoding video objects in a sequence of video frames including;

separately encoding shape for each of the objects,

separately encoding texture for each of the objects, and

while coding texture for a first object, evaluating the shape of the first object to determine whether a transformation block in the first object is transparent based on the shape for the first object, and if so, then skipping texture coding for the transformation block; and

separately decoding video objects in the sequence of video frames including;

separately decoding shape for each of the objects,

separately decoding texture for each of the objects, and

while decoding texture for a first object, evaluating whether a transformation block in the first object is transparent based on the shape of the first object, and if so, then skipping texture decoding for the transformation block.

2. The method of claim 1 wherein the steps of evaluating the shape are performed repeatedly to identify transparent transformation blocks within the first object and other objects in the sequence of video frames.

3. The method of claim 1 wherein the texture of the first object is intra-frame coded.

4. The method of claim 1 wherein the texture of the first object is inter-frame coded.

5. The method of claim 1 wherein the step of separately encoding video objects includes the following steps:

separately coding motion data for each of the objects,

while coding motion data for a first object, evaluating whether the transformation block is transparent, and if so, then skipping motion coding for the transformation block;

and wherein the step of separately decoding the video objects includes the following steps;

separately coding motion data for each of the objects,

while decoding the motion data for a first object, evaluating whether the transformation block is transparent, and if so, then skipping motion decoding for the transformation block.

6. The method of claim 1 further including:

while coding texture for a transformation block partially covered by the first object, evaluating the shape of the first object to determine whether a subtransformation block in the partially covered transformation block is transparent based on the shape of the first object, and if so, then skipping texture coding for the sub-transformation block.

7. The method of claim 1 further including:

while coding motion for a transformation block partially covered by the first object, evaluating the shape of the first object to determine whether a subtransformation block in the partially covered transformation block is transparent based on the shape of the first object, and if so, then skipping motion coding for the sub-transformation block.

8. The method of claim 1 further including:

while coding texture and motion for a transformation block partially covered by the first object, evaluating the shape of the first object to determine whether a sub-transformation block in the partially covered transformation block is transparent based on the shape of the first object, and if so, then skipping texture and motion coding for the sub- transformation block.

9. The method of claim 8 further including:

if the sub-transformation block is transparent, encoding a motion flag associated with the sub-transformation block indicating that there is no motion data associated with the sub-transformation block, and encoding a texture flag associated with the sub-transformation block indicating that there is no texture data associated with the sub-transformation block; and

while decoding the partially covered transformation block, reading the motion flag and the texture flag associated with sub-transformation blocks of the partially covered transformation block to determine whether the partially covered transformation block includes motion or texture data that requires decoding.

10. An object-based video coder for coding objects in a video sequence into a bit stream, where the objects comprise portions of video frames in the video sequence and are each associated with a bounding rectangle that encloses the objects in the video frames, and where the bounding rectangles are divided into transformation blocks, the encoder comprising:

a shape encoder for coding shape of the objects in the video sequence;

a motion encoder for computing motion estimation data for the objects in the video sequence, for computing error values between predicted objects and the objects in the video sequence, and for coding the motion estimation data for transformation blocks covered by the objects;

a texture encoder for encoding pixels comprising the objects for transformation blocks covered by the objects, and in communication with the motion encoder for encoding the error values for the transformation blocks covered by the objects;

wherein the texture encoder is operable to read the shape of the objects to identify the transparent transformation blocks and operable to skip the encoding of pixels and error values of the transparent transformation blocks; and

a multiplexor in communication with the shape, texture and motion coder for combining encoded shape, motion estimation and texture data into the bitstream.

11. The encoder of claim 10 wherein the motion coder is operable to read the shape of the objects to identify transparent transformation blocks in the bounding rectangles and is operable to skip the coding of motion estimation data for the transparent transformation blocks.

12. The encoder of claim 10 wherein the transformation blocks are divided into sub-transformation blocks, wherein the texture coder is operable to read the shape information of the objects to identify transparent subtransformation blocks, and wherein the texture coder is operable to skip texture coding of the transparent sub-transformation blocks.

13. An object-based video decoder for decoding a bitstream of compressed video objects into objects in a video sequence, where the objects comprise portions of video frames in the video sequence and are each associated with a bounding rectangle that encloses the objects in the video frames, and where the bounding rectangles are divided into transformation blocks, the decoder comprising:

a shape decoder for decoding shape of the compressed objects in the bitstream;

a texture decoder for decoding pixels from the compressed objects, and for decoding the error values for the objects;

wherein the texture decoder is operable to read the shape of the objects to identify the transparent transformation blocks and operable to skip the decoding of pixels and error values of the transparent transformation blocks

a motion decoder for computing motion estimation data for the objects in the video sequence, for computing error values between predicted objects and the objects in the video sequence, and for coding the motion estimation data for transformation blocks.

14. A computer readable medium on which is stored software for coding video data, which when executed by a computer, perform the steps of:

separately encoding video objects in a sequence of video frames including;

separately coding shape for each of the objects,

separately coding texture for each of the objects, and

while coding texture for a first object, evaluating whether a transformation block is covered by the shape of the first object based on the shape for the first object, and if not, then skipping texture coding for the transformation block.

15. In an object-based video coding method, a method for reducing coding overhead comprising:

separately encoding video objects in a sequence of video frames including;

encoding shape for each of the objects, including performing motion compensation on at least a first partially transparent transformation block in a first object,

setting a block transparency status flag corresponding to an entirely transparent sub-transformation block in the partially transparent transformation block,

encoding motion data for at least the first object, and

encoding texture for each of the objects; and

separately decoding video objects in the sequence of video frames including;

decoding motion data for at least the first object, including evaluating the block transparency flag, and skipping motion decoding for the entirely transparent sub-transformation block;

decoding shape for each of the objects,

decoding texture for each of the objects, and

reconstructing the sequence of video frames from the decoded shape, motion and texture data.

16. The method of claim 15 wherein the step of separately encoding video objects includes:

encoding shape, texture, and motion vectors in transformation blocks for each of a plurality of the objects in the video sequence, wherein at least some of the transformation blocks are partially transparent transformation blocks;

setting a block transparency status flag associated with sub-transformation blocks within the partially transparent transformation blocks to indicate which sub-transformation blocks within the partially transparent transformation blocks, if any, are entirely transparent; and

wherein the step of separately decoding the video objects includes;

evaluating the block transparency status flags for the partially transparent transformation blocks; and

skipping decoding of the motion vectors for sub-transformation blocks for which a corresponding block transparency flag is set.

17. In an object-based video coding method, a method for reducing coding overhead comprising:

separately encoding video objects in a sequence of video frames including;

separately encoding shape for each of the objects,

separately encoding texture for each of the objects, and

while coding texture for a transformation block partially covered by a first object, evaluating the shape of the first object to determine whether a sub-transformation block in the partially covered transformation block is transparent based on the shape of the first object, and if so, then skipping texture coding for the sub-transformation block and encoding a coded block pattern bit or bits only for non-transparent sub-transformation blocks in the partially covered transformation block; and

separately decoding video objects in the sequence of video frames including;

separately decoding shape for each of the objects,

separately decoding texture for each of the objects, and

while decoding texture for a transformation block partially covered by the first object, evaluating the shape of the first object to determine whether a sub-transformation block in the partially covered transformation block is transparent based on the shape of the first object, and if so, then skipping texture decoding for the sub-transformation block.

18. The method of claim 17 wherein coded block pattern bit or bits comprise a variable length code selected from one of three variable length code tables where each of the variable length code tables correspond to the number of non-transparent sub-transformation blocks in the transformation block.