Method for encoding and transcoding multiple video objects with variable temporal resolution

• US 6,650,705 B1
• Filed: 05/26/2000
• Issued: 11/18/2003
• Est. Priority Date: 05/26/2000
• Status: Expired due to Fees

• Current Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Inventors: Vetro, Anthony, Sun, Huifang
• Primary Examiner(s): Rao, Andy
• Application Number:
  US09/579,889
• Time in Patent Office: 1,271 Days
• Field of Search: 375/240-1-2
• US Class Current: 375/240.08
• CPC Class Codes: G06T 9/20 Contour coding, e.g. using detection of edges
• CPC Class Codes: H04N 19/21 with binary alpha-plane coding for video objects, e.g. context-based arithmetic encoding [CAE]
• CPC Class Codes: H04N 19/29 involving scalability at the object level, e.g. video object layer [VOL]
• CPC Class Codes: H04N 19/31 in the temporal domain
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream

Abstract

A video is first partitioned into video objects. If the video is uncompressed, then the partitioning is done with segmentation planes. In the case where the video is compressed, a demultiplexer is used for the partitioning. Over time, shape features are extracted from each partitioned object. The extracted shape features are combined to determined a temporal resolution for each object over time. The temporal resolutions are subsequently used to encode or transcode the video objects as an output compressed video.

Claims

1. A method for coding a video, comprising:

partitioning the video into a plurality of objects;

measuring, over time, differences in shape of each object to determine a plurality of shape features of each object;

combining, the plurality of shape features to determine a temporal resolution for each object; and

coding each object according to the corresponding temporal resolution of the object.

2. The method of claim 1 wherein the video is uncompressed data, and the partitioning, combining, and coding is performed in an encoder.

3. The method of claim 1 wherein the video is compressed data, and the partitioning, combining, and coding is performed in a transcoder.

4. The method of claim 1 wherein at least two objects are coded according to different corresponding temporal resolutions.

5. The method of claim 1 wherein the temporal resolution of a coded object is proportional to the shape difference associated with the coded object.

6. The method of claim 1 wherein the shape difference is a Hamming distance which measures the number of pixels that are different between the objects.

7. The method of claim 3 wherein the partitioned objects have binary shapes, and the Hamming distance, d, is defined as:

$d=\underset{n=0}{\overset{N-1}{\sum <br><br>}}<br><br>\text{\hspace{1em}<br><br>}<br><br>\underset{m=0}{\overset{M-1}{\sum <br><br>}}<br><br>\text{\hspace{1em}<br><br>}<br><br><br><br>{\mathrm{\alpha <br><br>}}_1<br><br>\left(m,n\right)-{\mathrm{\alpha <br><br>}}_2<br><br>\left(m,n\right)<br><br>$

where α

₁(m,n) and α

₂(m,n) are corresponding segmentation planes at different time instants.

8. The method of claim 3 wherein the shape difference is a Hausdorff distance, which defines as maxmin function between sets of pixels, associated with the objects.

9. The method of claim 8 wherein the maxmin function is

10. The method of claim 1 wherein the video includes a plurality of frames, and each frame includes a plurality of macroblocks, and the macroblocks are coded as opaque blocks, transparent blocks, and boundary blocks.

11. The method of claim 1 further comprising:

coding the shape features of the objects as meta-data.

12. The method of claim 1 further comprising;

extracting, overtime, a motion feature from each object;

combining, over time, the motion features with the shape features to determine the temporal resolution for each object over time.

13. The method of claim 1 further comprising;

extracting, overtime, a coding complexity from each object;

combining, over time, the coding complexity with the shape features to determine the temporal resolution for each object over time.

14. The method of claim 1 wherein the shape features of the objects are extracted from a plurality of cue levels of the video.

15. An apparatus for coding a video, comprising:

means for partitioning a video into a plurality of objects;

means for measuring, over time, differences in shape of each object to determine a plurality of shape features of each object;

means for combining, the plurality of shape features to determine a temporal resolution for each object; and

means for coding each object according to the corresponding temporal resolution of the object.

16. The apparatus of claim 15 wherein the means for partitioning and measuring includes a shape code, a motion estimator, a motion compensator and a texture coder.

17. The apparatus of claim 15 wherein the objects and shape features are stored in a memory.

18. The apparatus of claim 15 wherein the video is an uncompressed, and the means for combining is a rate control unit.

19. The apparatus of claim 15 wherein the video is compressed, and the means for combining is a transcoding control unit.