Minimum-delay jitter smoothing device and method for packet video communications

• US 5,534,937 A
• Filed: 04/14/1994
• Issued: 07/09/1996
• Est. Priority Date: 04/14/1994
• Status: Expired due to Term

• Current Assignee: Motorola Mobility LLC (Lenovo Group Ltd.)
• Original Assignee: Motorola, Inc. (Motorola Solutions, Inc.)
• Inventors: Zhu, Qin-Fan, Yong, Mei
• Primary Examiner(s): Kostak, Victor R.
• Application Number:
  US08/227,438
• Time in Patent Office: 817 Days
• Field of Search: 348/461, 348/463, 348/465, 348/466, 348/467, 348/497, 370/102
• US Class Current: 375/240.27
• CPC Class Codes: H04L 2012/5616 Terminal equipment, e.g. codecs, synch.
• CPC Class Codes: H04L 2012/5649 Cell delay or jitter
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 21/23406 involving management of server-side video buffer
• CPC Class Codes: H04N 21/236 Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream multiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00
• CPC Class Codes: H04N 21/434 Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream demultiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00
• CPC Class Codes: H04N 21/44004 involving video buffer management, e.g. video decoder buffer or video display buffer
• CPC Class Codes: H04Q 11/0478 Provisions for broadband connections

Abstract

The method and device of the present invention provide a mechanism at a network node to compensate for variable delays or delay jitters of video packets transported over a packet-switched network, such as an ATM network, and generate a continuous bitstream to an external decoder. The received video packets with variable delays are first depacketized and stored in a video information buffer and then are sent to the external decoder through another constant or variable bit-rate channel. Based on a hypothetical decoder buffer verifier condition, stuffing bits are inserted into the output video bitstream to prevent the decoder buffer from overflowing. Stuffing bits are sent if one of the following two cases occurs: A) the condition of the hypothetical decoder buffer verifier is violated; B) the video information buffer is empty. A minimum number of stuffing bits are sent each time to minimize the incurred delay and concurrently make the following three conditions satisfied: 1) overflow of the decoder buffer is avoided; 2) idleness of the channel is avoided; 3) the stuffing bits must comply with the syntax of the video bitstream.

Claims

1. A device in a receiving node of a packet video communication system for converting a received digital video packet stream into a continuous bitstream for transmission to a video decoder, the device comprising:

(A) a video depacketizer, operably coupled to receive and depacketize digital video packets of the received digital video packet stream into video information bits;

(B) a video information buffer, operably coupled to the video depacketizer for storing the video information bits;

(C) a decoder buffer verifier (DBV), operably coupled to the video information buffer, for determining a number of transmitted information bits for a video frame, and for monitoring a fullness of a hypothetical decoder buffer (HDB);

(D) a bitstream generator, operably coupled to the video information buffer and to the DBV, for transmitting the video information bits from the video information buffer at a predetermined rate and for inserting a number of stuffing bits into the video information bitstream wherein the number of stuffing bits is determined according to a predetermined scheme for minimizing buffering jitter to provide a continuous bitstream with a substantially minimized smoothing delay.

2. The device of claim 1 wherein the predetermined scheme includes inserting the stuffing bits into the bitstream when one of:

(A) a predetermined hypothetical decoder buffer condition is violated; and

(B) the video information buffer is empty.

3. The device of claim 1 wherein the fullness, B_n, of the hypothetical decoder buffer (HDB) at time t_n is determined according to B_n =B_n-1 +Rn_-1,n, where B_n-1 is the buffer fullness at time t_n-1, and R_n-1,n is the number of video information bits transmitted during frame interval n-1 from time t_n-1 to t_n.

4. The device of claim 2 wherein the predetermined hypothetical decoder buffer condition is violated when E_n <

B_n-1 +R_n-1,n+1 -S, where E_n is the number of information bits removed from the HDB for frame n, B_n-1 is the HDB fullness at time t_n-1, R_n-1,n+1 is the number of information bits transmitted from time t_n-1 to t_n+1, and S is the size of the HDB.

5. The device of claim 4 wherein the predetermined hypothetical buffer condition is checked once every frame interval, and when the predetermined hypothetical buffer condition is violated, the stuffing bits are output to the bitstream generator before sending any video information bits from the video information buffer.

6. The device of claim 5 wherein the minimum number of stuffing bits S_n is determined by S_n =B_n-1 +R_n-1,n+1 -S-E_n.

7. The device of claim 1 wherein the video decoder is an H.261 decoder.

8. The device of claim 7 wherein the bits stuffing is performed utilizing one of:

(A) macroblock address stuffing; and

(B) error correction framing.

9. The device of claim 1 wherein a predetermined rate for reading video information bits from the video information buffer is selectable.

10. The device of claim 1 wherein the decoder buffer verifier includes:

(A) a frame bit counter, operably coupled to the video information buffer, for counting a number of bits for each video frame E_n,

(B) a HDB fullness counter, operably coupled to the video information buffer, for counting a HDB fullness B_n,

(C) a buffer condition verifier, operably coupled to the frame bit counter and to the HDB fullness counter, for verifying whether the number of the bits for a preselected frame removed from the HDB satisfies a predetermined hypothetical decoder buffer condition, and, where the predetermined hypothetical decoder buffer condition is violated, for providing to the bitstream generator a number equal to the number of stuffing bits S_n.

11. The device of claim 1 wherein the bitstream generator includes:

(A) a stuffing bits generator, operably coupled to the video information buffer, to the buffer condition verifier, and to a stuffing bit buffer, for generating stuffing bits in accordance with the predetermined scheme,

(B) the stuffing bits buffer, operably coupled to the stuffing bits generator, for storing the generated stuffing bits and for providing the stuffing bits to an output switch in accordance with occurrence of one of selected predetermined occurrences,

(C) the output switch, operably coupled to the video information buffer and to the stuffing bits buffer, for receiving one of;

(C1) input from the stuffing bits buffer upon occurrence of one of the selected predetermined occurrences, and

(C2) input from the video information buffer upon nonoccurrence of one of the selected predetermined occurrences.

12. A method in a network receiving node of a packet video communication system for converting a received digital video packet bitstream into a continuous bitstream for transmission to a video decoder, the method comprising the steps of:

(A) receiving the digital video packet stream and storing the information bits contained in said packets;

(B) monitoring a fullness of a hypothetical decoder buffer by a decoder buffer verifier (DBV); and

(C) reading, by a bitstream generator, video information bits from the video information buffer at a predetermined rate and inserting, by the bitstream generator, a number of stuffing bits into the video information bitstream wherein the number of stuffing bits is determined according to a predetermined scheme for minimizing buffering jitter to provide a continuous bitstream with a substantially minimized smoothing delay.

13. The method of claim 12 wherein the predetermined scheme includes inserting the stuffing bits into the bitstream when one of:

(A) a predetermined hypothetical buffer condition is violated; and

(B) the video information buffer is empty.

14. The method of claim 12 wherein the fullness Bn of a hypothetical decoder buffer (HDB) at time tn, which is just before frame n is removed from the hypothetical decoder, is determined according to Bn=Bn-1+Rn-1,n, where Bn-1 is the buffer fullness just after frame n-1 is removed from the hypothetical decoder, and Rn-1,n is the number of video information bits transmitted during frame interval n-1 from time tn-1 to tn.

15. The method of claim 13 wherein the predetermined hypothetical decoder buffer condition is violated when E_n <

B_n-1 +R_n-1,n+1 -S, where E_n is the number of information bits removed from the hypothetical decoder buffer for frame n, B_n-1 is the hypothetical decoder buffer fullness at time tn-1, R_n-1,n+1 is the number of information bits transmitted from time tn-1 to tn+1, and S is the size of the hypothetical decoder buffer.

16. The method of claim 15 wherein the predetermined hypothetical buffer condition is checked at every frame interval, and when the predetermined hypothetical buffer condition is violated, the stuffing bits are output to the bitstream generator before sending any video information bits from the video information buffer.

17. The method of claim 16 wherein the minimum number of stuffing bits S_n is determined by S_n =B_n-1 +R_n-1,n+1 -S-E_n.

18. The method of claim 12 wherein the external decoder is an H.261 decoder.

19. The method of claim 18 wherein the bits stuffing is performed utilizing one of:

(A) macroblock address stuffing; and

(B) error correction framing.

20. The method of claim 12 wherein a predetermined rate for reading video information bits from the video information buffer is selectable.

21. The method of claim 12 wherein the step of monitoring a fullness of a hypothetical decoder buffer by a decoder buffer verifier (DBV)includes:

(A) utilizing a frame bit counter for counting a number of bits for each frame En transmitted from the video information buffer,

(B) utilizing a decoder buffer fullness counter for counting a decoder buffer fullness Bn,

(C) utilizing a buffer condition verifier for verifying whether the number of bits for a preselected frame removed from the HDB satisfies a predetermined hypothetical decoder buffer condition, and, when the predetermined hypothetical decoder buffer condition is violated, providing to the bitstream generator a number equal to the number of stuffing bits Sn.

22. The method of claim 12 wherein the step of reading, by a bitstream generator, video information bits from the video information buffer at a predetermined rate and inserting, by the bitstream generator, a number of stuffing bits into the video information bitstream wherein the number of stuffing bits is determined according to a predetermined scheme to provide substantially minimized delay jitter smoothing includes:

(A) utilizing a stuffing bits generator for generating stuffing bits in accordance with the predetermined scheme,

(B) utilizing a stuffing bits buffer for storing the generated stuffing bits and providing the stuffing bits to an output switch in accordance with occurrence of one of selected predetermined occurrences,

(C) utilizing the output switch for receiving one of;

(C1) input from the stuffing bits buffer upon occurrence of one of the selected predetermined occurrences, and

(C2) input from the video information buffer upon nonoccurrence of one of the selected predetermined occurrences.

23. The method of claim 22 wherein the selected predetermined occurrences include:

(A) where a frame is removed from the HDB, a nonzero Sn inputted from the buffer condition verifier, the stuffing bits generator generates Sn stuffing bits in compliance with a preselected decoder bitstream syntax, and where a number of the generated stuffing bits S_n is different from Sn, S_n is selected to be a smallest number that is no less than Sn in order to reduce incurred delay at a decoder, and

(B) where both the video information buffer and the stuffing bits buffer are empty, the stuffing bits generator generates a minimum number of stuffing bits that is in compliance with the preselected decoder bitstream syntax to ensure that channel will not be idle.

24. A hypothetical buffer-based jitter smoothing device for minimizing smoothing delay in a packet video communication system, the device comprising:

A) a video depacketizer, operably coupled to receive and depacketize digital video packets of a received digital video packet stream into video information bits,

B) an information buffer, operably coupled to the depacketizing means, for storing the video information bits,

C) hypothetical buffer-based smoothing means, operably coupled to the information buffer, for utilizing a hypothetical buffer to generate and transmit a continuous stuffed video information bitstream at a predetermined rate and minimize buffering jitter.

25. The hypothetical buffer-based jitter smoothing device of claim 24 wherein a number of stuffing bits inserted into the video information bitstream is determined according to a predetermined scheme.

26. The hypothetical buffer-based jitter smoothing device of claim 24 wherein the hypothetical buffer-based smoothing means determines a number of stuffing bits to be transmitted for a video frame and monitors a fullness of the hypothetical buffer.

27. The hypothetical buffer-based jitter smoothing device of claim 25 wherein the predetermined scheme includes inserting the stuffing bits into the bitstream, i.e., bits stuffing, when one of:

(A) a predetermined hypothetical buffer condition is violated; and

(B) the video information buffer is empty.

28. The hypothetical buffer-based jitter smoothing device of claim 26 wherein the fullness, B_n, of the hypothetical buffer at time t_n is determined according to B_n =B_n-1 +Rn₃₁ 1 ,n, where B_n-1 is the buffer fullness at time t_n-1, and R_n-1,n is the number of video information bits transmitted during frame interval n-1 from time t_n-1 to t_n.

29. The hypothetical buffer-based jitter smoothing device of claim 27 wherein the predetermined hypothetical buffer condition is violated when E_n <

B_n-1 +R_n-1,n+1 -S, where E_n is the number of information bits removed from the hypothetical buffer for frame n, B_n-1 is the fullness of said buffer at time t_n-1, R_n-1,n+1 is the number of information bits transmitted from time t_n-1 to t_n+1, and S is the size of the hypothetical buffer.

30. The hypothetical buffer-based jitter smoothing device of claim 27 wherein the predetermined hypothetical buffer condition is checked once every frame interval, and when the predetermined hypothetical buffer condition is violated, the stuffing bits are inserted into the video information bitstream before sending any video information bits from the video information buffer.

31. The hypothetical buffer-based jitter smoothing device of claim 30 wherein the minimum number of stuffing bits S_n is determined by S_n =B_n-1 +R_n-1,n+1 -S-E_n.

32. The hypothetical buffer-based jitter smoothing device of claim 24 wherein the continuous stuffed video information bitstream output from the jitter smoothing device is transmitted to an H.261 decoder.

33. The hypothetical buffer-based jitter smoothing device of claim 32 wherein the bits stuffing is performed utilizing one of:

(A) macroblock address stuffing; and

(B) error correction framing.

34. The hypothetical buffer-based jitter smoothing device of claim 26 wherein determining the number of stuffing bits to be transmitted for the video frame is performed at a predetermined selectable rate.

35. The hypothetical buffer-based jitter smoothing device of claim 24 wherein the hypothetical buffer-based smoothing means includes:

(A) a frame bit counter, operably coupled to the video information buffer, for counting a number of bits for each video frame E_n,

(B) a hypothetical buffer fullness counter, operably coupled to the video information buffer, for counting a hypothetical buffer fullness B_n,

(C) a buffer condition verifier, operably coupled to the frame bit counter and to the hypothetical buffer fullness counter, for verifying whether the number of the bits for a preselected frame removed from the hypothetical buffer satisfies a predetermined hypothetical buffer condition, and, where the predetermined hypothetical buffer condition is violated, for providing to the bitstream generator a number equal to the number of stuffing bits S_n,

(D) a stuffing bits generator, operably coupled to the video information buffer, to the buffer condition verifier, and to a stuffing bit buffer, for generating stuffing bits in accordance with the predetermined scheme,

(E) the stuffing bits buffer, operably coupled to the stuffing bits generator, for storing the generated stuffing bits and for providing the stuffing bits to an output switch in accordance with occurrence of one of selected predetermined occurrences, and

(F) the output switch, operably coupled to the video information buffer and to the stuffing bits buffer, for receiving one of;

(F1) input from the stuffing bits buffer upon occurrence of one of the selected predetermined occurrences, and

(F2) input from the video information buffer upon nonoccurrence of one of the selected predetermined occurrences.