Method and system of processing image sequences

• US 10,205,980 B2
• Filed: 02/15/2016
• Issued: 02/12/2019
• Est. Priority Date: 08/28/2001
• Status: Expired due to Fees

• Current Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Costa, Pierre
• Primary Examiner(s): An, Shawn S
• Application Number:
  US15/043,700
• Publication Number:
  US 20160165272A1
• Time in Patent Office: 1,093 Days
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/13 Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 19/169 characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/188 the unit being a video data packet, e.g. a network abstraction layer [NAL] unit
• CPC Class Codes: H04N 19/23 with coding of regions that are present throughout a whole video segment, e.g. sprites, background or mosaic
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/88 involving rearrangement of data among different coding units, e.g. shuffling, interleaving, scrambling or permutation of pixel data or permutation of transform coefficient data among different blocks
• CPC Class Codes: H04N 19/91 Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
• CPC Class Codes: H04N 21/23406 involving management of server-side video buffer
• CPC Class Codes: H04N 21/2343 involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/23439 for generating different versions
• CPC Class Codes: H04N 21/23605 Creation or processing of packetized elementary streams [PES]
• CPC Class Codes: H04N 21/23611 Insertion of stuffing data into a multiplex stream, e.g. to obtain a constant bitrate synchronisation arrangements in time-division multiplex systems using bit stuffing for systems with different or fluctuating information rates H04J3/073
• CPC Class Codes: H04N 21/23805 Controlling the feeding rate to the network, e.g. by controlling the video pump
• CPC Class Codes: H04N 21/2383 Channel coding or modulation of digital bit-stream, e.g. QPSK modulation arrangements for detecting or preventing errors in the information received by adapting the channel coding H04L1/0009; analogue front ends or means for connecting modulators, demodulators or transceivers to a transmission line H04L27/0002
• CPC Class Codes: H04N 21/2385 Channel allocation H04N21/266 takes precedence; Bandwidth allocation H04N21/24 takes precedence ; allocation of channels according to the instantaneous demands of the users in time-division multiplex systems H04J3/1682; admission control, resource allocation in open networks H04L12/5692; arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896; negotiating bandwidth in wireless networks H04W28/16
• CPC Class Codes: H04N 21/2662 : Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/4305 : Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets arrangements for synchronising receiver with transmitter by comparing receiver clock with transmitter clock H04L7/0012; arrangements for synchronising receiver with transmitter wherein the receiver takes measures against momentary loss of synchronisation H04L7/0083
• CPC Class Codes: H04N 21/4343 : Extraction or processing of packetized elementary streams [PES]
• CPC Class Codes: H04N 21/4346 : involving stuffing data, e.g. packets or bytes synchronisation arrangements in time-division multiplex systems with different or fluctuating information rates H04J3/073
• CPC Class Codes: H04N 21/44004 : involving video buffer management, e.g. video decoder buffer or video display buffer

Abstract

A system or method for processing a variable bit rate representation of an image sequence can include segmenting a variable bit rate representation of an image sequence into a plurality of segments and creating a second representation of the image sequence where a block of information from a first segment of the plurality of segments is interlaced with blocks of information of a second segment of the plurality of segments. The second representation of the image sequence can include a fragment header indicating the block of information of the first segment of the plurality of segments interlaced with the blocks of information of the second segment of the plurality of segments.

Claims

1. A method, comprising:

segmenting, by a processing system comprising a processor, a variable bit rate representation of an image into a first plurality of packets and a second plurality of packets, wherein the first plurality of packets spans a first time interval, wherein the second plurality of packets spans a second time interval, and wherein a first amount of time corresponding to the first time interval is different than a second amount of time corresponding to the second time interval;

identifying, by the processing system, a first packet of the first plurality of packets, wherein the first packet includes a first number of blocks of information greater than a baseline value;

removing, by the processing system, a group of blocks from the first number of blocks of information from the first packet, wherein a remaining group of blocks from the first number of blocks is less than the baselines value;

identifying, by the processing system, a second packet and a third packet from the first plurality of packets, wherein the second packet includes a second number of blocks of information less than the baseline value, and wherein the third packet includes a third number of blocks of information less than the baseline value;

adding, by the processing system, a first portion of the group of blocks to the second packet resulting in an adjusted second number of blocks of information and adding a second portion of the group of blocks to the third packet resulting in an adjusted third number of blocks of information, wherein the adjusted second number of blocks is less than the baseline value and the adjusted third number of blocks is less than the baseline value; and

creating, by the processing system, a second representation of the variable bit rate representation based on the removing of the group of blocks from the first packet and the adding of the first portion of the group of the blocks to the second packet and adding the second portion of the group of blocks to the third packet to cause a first size of the first packet, a second size of the second packet, and a third size of the third packet to be within an upper bound after creating the second representation, wherein each packet of the second representation comprises reconstruction information to describe a reconstruction of the variable bit rate representation from the second representation; and

streaming, by the processing system, the second representation of the variable bit rate representation to a device.

2. The method of claim 1, wherein the second representation of the variable bit rate representation that is streamed provides an approximately constant bit rate representation of image date to the device.

3. The method of claim 1, wherein the reconstruction information comprises block number data identifying an original packet of the group of blocks that are removed from the first packet.

4. The method of claim 1, wherein the reconstruction information comprises fragment number data to identify the group of blocks that are removed from the first packet.

5. The method of claim 1, wherein the reconstruction information comprises length data indicating a length of the group of blocks that are removed from the first packet.

6. The method of claim 1, wherein the reconstruction information comprises last fragment number data to indicate a sequence of the group of blocks that are removed from the first packet.

7. The method of claim 1, wherein the reconstruction information comprises optional stuffing length data for the group of blocks that are removed from the first packet.

8. The method of claim 1, wherein the reconstruction information comprises peak size data indicating how many time-advance bytes are need to be buffered to reconstruct the variable bit rate representation.

9. The method of claim 1, wherein, after creating the second representation, the first packet and the second packet comprise a same number of blocks of information.

10. The method of claim 1, wherein the device that receives the second representation reconstructs frames of the variable bit rate representation from the second representation.

11. The method of claim 10, wherein the device that receives the second representation performs reconstructions of frames of the variable bit rate representation based on reconstruction information and blocks of information received in real time.

12. A method, comprising:

segmenting, by a processing system including a processor, a variable bit rate representation of an image into a first plurality of packets and a second plurality of packets, wherein the first plurality of packets spans a first time interval, wherein the second plurality of packets spans a second time interval, and wherein a first amount of time corresponding to the first time interval is different than a second amount of time corresponding to the second time interval;

identifying, by the processing system, a first packet of the first plurality of packets, wherein the first packet includes a first number of blocks of information greater than a baseline value;

removing, by the processing system, a group of blocks from the first number of blocks of information from the first packet, wherein a remaining group of blocks from the first number of blocks is less than the baselines value;

identifying, by the processing system, a second packet and a third packet from the first plurality of packets, wherein the second packet includes a second number of blocks of information less than the baseline value, and wherein the third packet includes a third number of blocks of information less than the baseline value;

adding, by the processing system, a first portion of the group of blocks to the second packet resulting in an adjusted second number of blocks of information and adding a second portion of the group of blocks to the third packet resulting in an adjusted third number of blocks of information, wherein the adjusted second number of blocks is less than the baseline value and the adjusted third number of blocks is less than the baseline value; and

creating, by the processing system, a second representation of the variable bit rate representation based on the removing of the group of blocks from the first packet and the adding of the first portion of the group of the blocks to the second packet and adding the second portion of the group of blocks to the third packet to cause a first size of the first packet, a second size of the second packet, and a third size of the third packet to be within an upper bound after creating the second representation.

13. The method of claim 12, wherein each packet of the second representation comprises reconstruction information to describe a reconstruction of the variable bit rate representation from the second representation, wherein the reconstruction information comprises peak size data indicating how many time-advance bytes are need to be buffered to reconstruct the variable bit rate representation.

14. A non-transitory computer-readable storage medium, comprising executable instructions that, when executed by a processing system including a processor, facilitate performance of operations, comprising:

segmenting a variable bit rate representation of an image into a first plurality of packets and a second plurality of packets, wherein the first plurality of packets spans a first time interval, wherein the second plurality of packets spans a second time interval, and wherein a first amount of time corresponding to the first time interval is different than a second amount of time corresponding to the second time interval;

identifying a first packet of the first plurality of packets, wherein the first packet includes a first number of blocks of information greater than a baseline value;

removing a group of blocks from the first number of blocks of information from the first packet, wherein a remaining group of blocks from the first number of blocks is less than the baselines value;

identifying a second packet and a third packet from the first plurality of packets, wherein the second packet includes a second number of blocks of information less than the baseline value, and wherein the third packet includes a third number of blocks of information less than the baseline value;

adding a first portion of the group of blocks to the second packet resulting in an adjusted second number of blocks of information and adding a second portion of the group of blocks to the third packet resulting in an adjusted third number of blocks of information, wherein the adjusted second number of blocks is less than the baseline value and the adjusted third number of blocks is less than the baseline value; and

creating a second representation of the variable bit rate representation based on the removing of the group of blocks from the first packet and the adding of the first portion of the group of the blocks to the second packet and adding the second portion of the group of blocks to the third packet to cause a first size of the first packet, a second size of the second packet, and a third size of the third packet to be within an upper bound after creating the second representation; and

streaming the second representation of the variable bit rate representation to a device.

15. The non-transitory computer-readable storage medium of claim 14, wherein each packet of the second representation comprises reconstruction information to describe a reconstruction of the variable bit rate representation from the second representation.

16. The non-transitory computer-readable storage medium of claim 15, wherein the second representation of the variable bit rate representation that is streamed provides an approximately constant bit rate representation of image date to the device.

17. The non-transitory computer-readable storage medium of claim 15, wherein the reconstruction information comprises block number data identifying an original packet of the group of blocks that are removed from the first packet.

18. The non-transitory computer-readable storage medium of claim 15, wherein the reconstruction information comprises fragment number data to identify the group of blocks that are removed from the first packet.

19. The non-transitory computer-readable storage medium of claim 15, wherein the reconstruction information comprises length data indicating a length of the group of blocks that are removed from the first packet.

20. The non-transitory computer-readable storage medium of claim 15, wherein the reconstruction information comprises last fragment number data to indicate a sequence of the group of blocks that are removed from the first packet, and wherein the reconstruction information comprises peak size data indicating how many time-advance bytes are need to be buffered to reconstruct the variable bit rate representation.