Robust, reliable compression and packetization scheme for transmitting video

• US 6,680,976 B1
• Filed: 05/09/2000
• Issued: 01/20/2004
• Est. Priority Date: 07/28/1997
• Status: Expired due to Term

• Current Assignee: Board of Trustees of The University of Illinois
• Original Assignee: Board of Trustees of The University of Illinois
• Inventors: Chen, Zhigang, Campbell, Roy H.
• Primary Examiner(s): BRITTON, HOWARD W
• Application Number:
  US09/463,686
• Time in Patent Office: 1,351 Days
• Field of Search: H04/N.736, H04/N.764, G06/F 1.300, 348/387.1, 382/332, 375/240.26, 375/240.12
• US Class Current: 375/240.26
• CPC Class Codes: H04N 19/517 by encoding
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/63 using sub-band based transform, e.g. wavelets
• CPC Class Codes: H04N 19/647 using significance based coding, e.g. Embedded Zerotrees of Wavelets [EZW] or Set Partitioning in Hierarchical Trees [SPIHT]
• CPC Class Codes: H04N 19/65 using error resilience
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 19/89 involving methods or arrangements for detection of transmission errors at the decoder
• CPC Class Codes: H04N 21/2383 Channel coding or modulation of digital bit-stream, e.g. QPSK modulation arrangements for detecting or preventing errors in the information received by adapting the channel coding H04L1/0009; analogue front ends or means for connecting modulators, demodulators or transceivers to a transmission line H04L27/0002
• CPC Class Codes: H04N 21/6473 Monitoring network processes errors for recovering from a failure of a protocol instance or entity H04L69/40

Abstract

For transmitting digital video information over a network such as the Internet, a hybrid coding scheme (FIGS. 9-18) with wavelet I frame coding and H.263 predictive coding produces a robust, low bit rate video coding scheme. A new packetization method analyzes macroblock level dependency structure (FIG. 19) and packetizes the bit stream so as to minimize dependencies between packets and thereby minimize packet loss damage.

Claims

1. A method of transmitting video including a sequence of frames each containing a plurality of blocks, comprising the steps of:

organizing said video into plural packets and transmitting said packets;

selecting blocks in at least two of said frames which have coding dependencies; and

transmitting said selected blocks within a single packet.

7. The method of claim 1, further comprising coding the sequence of frames to provide an encoded video sequence and packetizing the encoded video sequence to provide the packets.

8. The method of claim 1, further comprising reconstructing the video from the packets including substituting a received block for a first one of the frames of the sequence for a lost or skipped block of a second one of the frames of the sequence.

9. The method of claim 1, further comprising evaluating interframe dependency from a plurality of the blocks selected from two or more frames and identifying a moving image object from said evaluating.

10. The method of claim 1, wherein said transmitting is performed on a network subject to loss of one or more of the packets and delivery of one or more of the packets in an order different than said transmitting.

11. The method of claim 10, wherein the network is the internet.

2. A method, comprising:

providing video for transmission in a plurality of transmission packets, the video including a sequence of frames, the frames each including a plurality of blocks;

identifying a dependency between two or more blocks of different frames; and

transmitting the two or more blocks in a single one of the packets.

3. The method of claim 2, wherein the number of the blocks are selected based on a dependency tree.

4. The method of claim 3, further comprising sorting and merging the transmission packets based on the dependency tree.

5. The method of claim 2, wherein the frames are encoded into one of at least two different types including an I frame type and a P frame type, and the number of blocks are selected from two or more frames of the P frame type.

12. The method of claim 2, further comprising selecting the number of the blocks based on motion of an image object defined by the video.

13. The method of claim 2, further comprising reconstructing the video from the packets including substituting a received block for a first one of the frames of the sequence for a lost or skipped block of a second one of the frames of the sequence.

6. A method, comprising:

providing a sequence of several video frames for transmission;

classifying each of the video frames of the sequence into one of at least two different types, the different types including a first type and a second type;

coding the video frames of the first type with a wavelet transform-based coding scheme and the video frames of the second type with a DCT-based coding scheme to provide a coded video sequence; and

transmitting the coded video sequence.

14. The method of claim 6, wherein the video frames of the second type are each coded with reference to content of at least one other of the video frames.

15. The method of claim 14, wherein the video frames of the first type are each coded independent of any other of the video frames and the video frames of the first type require transmission of more information than the video frames of the second type.

16. The method of claim 15, wherein the first type of the video frames are I frames and the second type of the video frames are P frames.

17. The method of claim 6, wherein the video frames of the first type are fewer in number than the video frames of the second type for the sequence of the video frames, and said classifying includes designating at least two of the video frames of the second type between each of the video frames of the first type relative to the sequence of the video frames.

18. The method of claim 6, wherein said coding of the video frames of the second type includes compensating for image motion.

19. The method of claim 6, further comprising relating a number of wavelet transform coefficients with a zero tree during performance of said coding of the video frames of the first type.

20. The method of claim 19, wherein said relating includes comparing each of a number of nodes of the zero tree to a threshold.

21. The method of claim 6, wherein the video frames each include a plurality of blocks, said coding includes selecting a first block of a first one of the frames and a second block of a second one of the frames based on a dependency between the first block and the second block, and said transmitting includes transmitting the first block and the second block within a single transmission packet.

22. The method of claim 21, wherein the first one of the frames and the second one of the frames are both of the second type.

23. A video transmission system, comprising:

a sending device operable to encode video from a sequence of video frames into a plurality of transmission packets, the video frames each being comprised of a plurality of video blocks, said sending device being operable to identify a dependency between two or more of the video blocks of different frames of the sequence and transmit the two or more blocks in a common one of the transmission packets.

24. The system of claim 23, wherein said sending device is operable to select a number of the blocks based on a dependency tree.

25. The system of claim 24, wherein said sending device is operable to sort and merge the transmission packets based on the dependency tree.

26. The system of claim 23, wherein said sending device is operable to encode the frames into one of at least two different types including an I frame type and a P frame type, and the two or more blocks are selected from two or more frames of the P frame type.

27. The system of claim 23, wherein said sending device includes means for selecting the two or more blocks based on motion of an image object defined by the video.

28. The system of claim 23, wherein said sending device includes means for evaluating interframe dependency from a plurality of the blocks selected from two or more frames and identifying a moving image object.

29. The system of claim 23, further comprising a receiving device operable to reconstruct the video from the packets.

30. The system of claim 29, wherein said sending device and said receiving device are coupled together by a network.

31. The system of claim 30, wherein said network is subject to loss of one or more of the packets and delivery of one or more of the packets to said receiving device in a different order than transmitted by said sending device, and said receiving device includes means for substituting a missing block of one of the frames with a block from another of the frames.

32. The system of claim 31, wherein said network includes the intemet.

33. The system of claim 23, wherein said sending device is operable to encode a first frame type based on a first transform type and a second frame type based on a second transform type.

34. The system of claim 33, wherein said sending device is operable to encode the video frames of the first type independent of any other of the video frames and the video frames of the second type with reference to content of at least one other of the video frames.

35. The system of claim 33, wherein said sending device is operable to designate at least two of the video frames of the second type between each of the video frames of the first type relative to the sequence of the video frames.

36. The system of claim 33, wherein the first transform type is wavelet based and the second transform type is DCT based.

37. A method, comprising:

receiving a number of transmission packets from a computer network, the computer network being subject to loss of one or more packets;

providing a sequence of video frames based on the transmission packets, the frames each including a number of video blocks, said providing including;

determining two or more video blocks of a common one of the transmission packets are for different frames of the sequence, the two or more video blocks being dependent on one another; and

substituting a received video block for one of the frames in place of a missing video block of another of the frames.

38. The method of claim 37, wherein the transmission packets are received at a varying rate, and further comprising skipping one or more of the video frames to provide the sequence at a desired rate.

39. The method of claim 37, wherein said providing includes decoding the frames from at least two different types, a first type corresponding to a wavelet transform-based coding scheme and a second type corresponding to a DCT-based coding scheme.

40. The method of claim 37, wherein the number of the blocks for each of the transmission packets are selected based on a dependency tree.