Very low bit rate video coding/decoding method and apparatus

• US 6,084,912 A
• Filed: 06/27/1997
• Issued: 07/04/2000
• Est. Priority Date: 06/28/1996
• Status: Expired due to Term

• Current Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.), MediaTek, Inc.
• Original Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.), Sarnoff Corporation (SRI International, Inc.)
• Inventors: Zhang, Ya-Qin, Reitmeier, Glenn
• Primary Examiner(s): RAO, ANAND SHASHIKANT
• Application Number:
  US08/884,666
• Time in Patent Office: 1,103 Days
• Field of Search: 348/407, 348/399, 348/402, 348/699, 348/413, 348/416, 348/415, 382/276, 382/107, 382/268, 382/209, 375/340
• US Class Current: 375/240.11
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/146 Data rate or code amount at the encoder output
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/1883 the unit relating to sub-band structure, e.g. hierarchical level, directional tree, e.g. low-high [LH], high-low [HL], high-high [HH]
• CPC Class Codes: H04N 19/503 involving temporal prediction adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/63 using sub-band based transform, e.g. wavelets
• CPC Class Codes: H04N 19/647 using significance based coding, e.g. Embedded Zerotrees of Wavelets [EZW] or Set Partitioning in Hierarchical Trees [SPIHT]
• CPC Class Codes: H04N 19/94 Vector quantisation

Abstract

A method and apparatus for coding motion video (images) that produces very low bit rate video representation and minimizes the amount of data needed to be transferred to the decoder. The apparatus comprises a 2-pass segmentation section, a 2-stage motion estimation/compensation section that optionally employs Affine Motion Compensation (AMC), a wavelet transform section, a Zero-Tree Wavelet Quantization (ZTQ) section, a code book, a comparator, a reconstruction section, a coder, a frame memory and an entropy coder.

Claims

1. A method of encoding an input image to produce a very low bit rate bitstream, said method comprising the steps of:

(a) segmenting the input image into a background and at least one object;

(b) applying motion estimation (ME) and compensation (MC) to said segmented image to produce a compensated image;

(c) applying a transform to said compensated image to produce a transformed image;

(d) selectively updating a codebook by storing said transformed image or a portion of said transformed image into said codebook; and

(e) selectively encoding said transformed image or a portion of said transformed image into the bitstream.

2. The method of claim 1, wherein said segmenting step (a) comprises a first segmentation step and a second segmentation step.

3. The method of claim 2, wherein said second segmentation step is applied to a "segmentation-failure" region.

4. The method of claim 1, wherein said applying ME/MC step (b) comprises a first ME step and a second ME step.

5. The method of claim 4, wherein said second ME step is applied to said object.

6. The method of claim 4, wherein said applying ME/MC step (b) uses an affine motion ME/MC method.

7. The method of claim 6, wherein said affine motion ME/MC method classifies a motion block as "No Motion", "Motion Compensatable" or "Occluded".

8. The method of claim 1, wherein said selectively storing step (d) stores said transformed image of said background.

9. The method of claim 1, wherein said transform applying step (c) applies a wavelet transform and wherein said selectively encoding step (e) encodes only a LL band of said transformed image of said background.

10. A method of encoding an input image to produce a very low bit rate bitstream, said method comprising the steps of:

(a) selectively updating a codebook by storing a transformed image or a portion of said transformed image of the input image into said codebook;

(b) using index to represent a location of said stored transformed image or a portion of said stored transformed image within said codebook; and

(c) selectively encoding a portion of said transformed image and said index into a bitstream.

11. The method of claim 10, wherein said stored transformed image is a background of the input image.

12. The method of claim 11, wherein said encoding step (c) encodes a transformed image of an object of the input image and said index.

13. An apparatus for encoding an input image to produce a very low bit rate bitstream comprising:

a segmentation section for segmenting the input image into a background and at least one object;

a motion estimation (ME) and compensation (MC) section, coupled to said segmentation section, for applying motion estimation and compensation to said segmented image to produce a compensated image;

a spatial coder, coupled to said (ME) and (MC) section, for applying a transform to said compensated image to produce a transformed image;

a storage, coupled to said spatial coder, for selectively updating a codebook by storing said transformed image or a portion of said transformed image into said codebook; and

a coder, coupled to said storage, for encoding selectively said transformed image or a portion of said transformed image into the bitstream.

14. The apparatus of claim 13, wherein said segmentation section employs a first segmentation step and a second segmentation step.

15. The apparatus of claim 14, wherein said second segmentation step is applied to a "segmentation-failure" region.

16. The apparatus of claim 13, wherein said applying ME/MC section employs a first ME step and a second ME step.

17. The apparatus of claim 16, wherein said second ME step is applied to said object.

18. The apparatus of claim 16, wherein said ME/MC section employs an affine motion ME/MC method.

19. The apparatus of claim 18, wherein said affine motion ME/MC method classifies a motion block as "No Motion", "Motion Compensatable" or "Occluded".

20. The apparatus of claim 13, wherein said storage stores said transformed image of said background.