Method and system to improve the transport of compressed video data in real time

• US 7,075,990 B2
• Filed: 01/16/2003
• Issued: 07/11/2006
• Est. Priority Date: 08/28/2001
• Status: Expired due to Fees

• Current Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Original Assignee: SBC Properties, L.P.
• Inventors: Costa, Pierre
• Primary Examiner(s): An, Shawn S.
• Application Number:
  US10/346,727
• Publication Number:
  US 20030169810A1
• Time in Patent Office: 1,272 Days
• Field of Search: 375/240.24, 375/240.05, 375/240.23, 375/240.01, 375/240.25, 375/240.26, 375/240.27, 348/512, 348/419, 382/232, 382/246, 370/468, 370/235, 370/470, 370/472, 370/473, 370/476, 370/477
• US Class Current: 375/240.24
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/13 Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 19/169 characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/188 the unit being a video data packet, e.g. a network abstraction layer [NAL] unit
• CPC Class Codes: H04N 19/23 with coding of regions that are present throughout a whole video segment, e.g. sprites, background or mosaic
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/88 involving rearrangement of data among different coding units, e.g. shuffling, interleaving, scrambling or permutation of pixel data or permutation of transform coefficient data among different blocks
• CPC Class Codes: H04N 19/91 Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
• CPC Class Codes: H04N 21/23406 involving management of server-side video buffer
• CPC Class Codes: H04N 21/2343 involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/23439 for generating different versions
• CPC Class Codes: H04N 21/23605 Creation or processing of packetized elementary streams [PES]
• CPC Class Codes: H04N 21/23611 Insertion of stuffing data into a multiplex stream, e.g. to obtain a constant bitrate synchronisation arrangements in time-division multiplex systems using bit stuffing for systems with different or fluctuating information rates H04J3/073
• CPC Class Codes: H04N 21/23805 Controlling the feeding rate to the network, e.g. by controlling the video pump
• CPC Class Codes: H04N 21/2383 Channel coding or modulation of digital bit-stream, e.g. QPSK modulation arrangements for detecting or preventing errors in the information received by adapting the channel coding H04L1/0009; analogue front ends or means for connecting modulators, demodulators or transceivers to a transmission line H04L27/0002
• CPC Class Codes: H04N 21/2385 Channel allocation H04N21/266 takes precedence; Bandwidth allocation H04N21/24 takes precedence ; allocation of channels according to the instantaneous demands of the users in time-division multiplex systems H04J3/1682; admission control, resource allocation in open networks H04L12/5692; arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896; negotiating bandwidth in wireless networks H04W28/16
• CPC Class Codes: H04N 21/2662 : Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/4305 : Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets arrangements for synchronising receiver with transmitter by comparing receiver clock with transmitter clock H04L7/0012; arrangements for synchronising receiver with transmitter wherein the receiver takes measures against momentary loss of synchronisation H04L7/0083
• CPC Class Codes: H04N 21/4343 : Extraction or processing of packetized elementary streams [PES]
• CPC Class Codes: H04N 21/4346 : involving stuffing data, e.g. packets or bytes synchronisation arrangements in time-division multiplex systems with different or fluctuating information rates H04J3/073
• CPC Class Codes: H04N 21/44004 : involving video buffer management, e.g. video decoder buffer or video display buffer

Abstract

A variable bit rate (VBR) representation of an image sequence is segmented based on a plurality of time intervals. For each of at least two of the time intervals, the method comprises: determining which of the packets, denoted by Pp, in the VBR representation within the presently-considered time interval has a number of blocks of information per unit time greater than a baseline value; determining which of the packets, denoted by Pn, in the VBR representation within the presently-considered time interval has a number of blocks of information per unit time less than the baseline value; and creating a second representation of the image sequence in which some blocks of information Bp are removed from at least one Pp packet and interlaced with blocks of information in at least one Pn packet to produce reformatted packets.

Claims

1. A method comprising:

providing a variable bit rate (VBR) representation of an image sequence, the VBR representation comprising a plurality of packets of blocks of information;

segmenting the VBR representation based on a plurality of time intervals;

for each of at least two of the time intervals;

determining which of the packets, denoted by Pp, in the VBR representation within a presently-considered time interval has a number of blocks of information per unit time greater than a baseline value;

determining which of the packets, denoted by Pn, in the VBR representation within the presently-considered time interval has a number of blocks of information per unit time less than the baseline value; and

creating a second representation of the image sequence in which some blocks of information Bp are removed from at least one Pp packet and interlaced with blocks of information in at least one Pn packet to produce reformatted packets having a reduced variation in a number of blocks of information per unit time.

2. The method of claim 1 wherein each of the reformatted packets in the presently-considered time interval has the same size.

3. The method of claim 1 wherein each of the reformatted packets in at least two of the time intervals has the same size.

4. The method of claim 1 wherein at least one of the reformatted packets comprises a fragment header indicating a source of the blocks of information Bp.

5. The method of claim 1 wherein at least one of the reformatted packets comprises all of an original VBR packet, an associated packet header identifying the original VBR packet and having data indicating a length of the original VBR packet, a fragment of another original VBR packet, and a fragment header identifying the other original VBR packet and having data indicating a length of the fragment.

6. The method of claim 1 further comprising:

extending the length of a time interval so that each of the reformatted packets associated with the time interval has a size that is within an upper bound.

7. The method of claim 1 further comprising:

removing at least one frame from the image sequence so that each of the reformatted packets associated with a time interval has a size that is within an upper bound.

8. The method of claim 1 further comprising:

streaming the second representation of the image sequence via a communication network;

receiving the second representation of the image sequence via the communication network; and

reconstructing frames of the image sequence concurrently with said receiving, said reconstructing comprising;

for reformatted packets corresponding to the packets Pn, reconstructing frames of the image sequence based on blocks of information received about in real time, and storing the blocks of information Bp in a buffer; and

for reformatted packets corresponding to the packets Pp, reconstructing frames of the image sequence based on the blocks of information Bp stored in the buffer and blocks of information received about in real time.

9. A system to process a variable bit rate (VBR) representation of an image sequence, the VBR representation comprising a plurality of packets of blocks of information, the system comprising:

a processor to segment the VBR representation based on a plurality of time intervals, and for each of at least two of the time intervals;

to determine which of the packets, denoted by Pp, in the VBR representation within a presently-considered time interval has a number of blocks of information per unit time greater than a baseline value;

to determine which of the packets, denoted by Pn, in the VBR representation within the presently-considered time interval has a number of blocks of information per unit time less than the baseline value; and

to create a second representation of the image sequence in which some blocks of information Bp are removed from at least one Pp packet and interlaced with blocks of information in at least one Pn packet to produce reformatted packets having a reduced variation in a number of blocks of information per unit time.

10. The system of claim 9 wherein each of the reformatted packets in the presently-considered time interval has the same size.

11. The system of claim 9 wherein each of the reformatted packets in at least two of the time intervals has the same size.

12. The system of claim 9 wherein at least one of the reformatted packets comprises a fragment header indicating a source of the blocks of information Bp.

13. The system of claim 9 wherein at least one of the reformatted packets comprises all of an original VBR packet, an associated packet header identifying the original VBR packet and having data indicating a length of the original VBR packet, a fragment of another original VBR packet, and a fragment header identifying the other original VBR packet and having data indicating a length of the fragment.

14. The system of claim 9 wherein the processor is to extend the length of a time interval so that each of the reformatted packets associated with the time interval has a size that is within an upper bound.

15. The system of claim 9 wherein the processor is to remove at least one frame from the image sequence so that each of the reformatted packets associated with a time interval has a size that is within an upper bound.

16. The system of claim 9 further comprising:

a transmitter to stream the second representation of the image sequence via a communication network.

17. The system of claim 16 further comprising:

a receiver to receive the second representation of the image sequence via the communication network;

a buffer; and

a second processor responsive to the receiver to reconstruct frames of the image sequence concurrently with the second representation being received;

wherein for reformatted packets corresponding to the packets Pn, the second processor is to reconstruct frames of the image sequence based on blocks of information received about in real time, and to store the blocks of information Bp in the buffer; and

wherein for reformatted packets corresponding to the packets Pp, the second processor is to reconstruct frames of the image sequence based on the blocks of information Bp stored in the buffer and blocks of information received about in real time.

18. A computer-readable medium having computer program code to direct a processor to process a variable bit rate (VBR) representation of an image sequence, the VBR representation comprising a plurality of packets of blocks of information, the computer program code directing the processor to:

segment the VBR representation based on a plurality of time intervals; and

for each of at least two of the time intervals;

determine which of the packets, denoted by Pp, in the VBR representation within a presently-considered time interval has a number of blocks of information per unit time greater than a baseline value;

determine which of the packets, denoted by Pn, in the VBR representation within the presently-considered time interval has a number of blocks of information per unit time less than the baseline value; and

create a second representation of the image sequence in which some blocks of information Bp are removed from at least one Pp packet and interlaced with blocks of information in at least one Pn packet to produce reformatted packets having a reduced variation in a number of blocks of information per unit time.

19. The computer-readable medium of claim 18 wherein each of the reformatted packets in the presently-considered time interval has the same size.

20. The computer-readable medium of claim 18 wherein each of the reformatted packets in at least two of the time intervals has the same size.

21. The computer-readable medium of claim 18 wherein at least one of the reformatted packets comprises a fragment header indicating a source of the blocks of information Bp.

22. The computer-readable medium of claim 18 wherein at least one of the reformatted packets comprises all of an original VBR packet, an associated packet header identifying the original VBR packet and having data indicating a length of the original VBR packet, a fragment of another original VBR packet, and a fragment header identifying the other original VBR packet and having data indicating a length of the fragment.

23. The computer-readable medium of claim 18 wherein the computer program code further directs the processor to extend the length of a time interval so that each of the reformatted packets associated with the time interval has a size that is within an upper bound.

24. The computer-readable medium of claim 18 wherein the computer program code further directs the processor to remove at least one frame from the image sequence so that each of the reformatted packets associated with a time interval has a size that is within an upper bound.