Enhanced block-request streaming using cooperative parallel HTTP and forward error correction

• US 9,628,536 B2
• Filed: 10/08/2015
• Issued: 04/18/2017
• Est. Priority Date: 06/09/2006
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Luby, Michael, Wang, Bin, Watson, Mark, Vicisano, Lorenzo, Pakzad, Payam
• Primary Examiner(s): LAMARRE, GUY J
• Application Number:
  US14/878,694
• Publication Number:
  US 20160065640A1
• Time in Patent Office: 558 Days
• CPC Class Codes: H03M 13/05 using block codes, i.e. a predetermined number of check bits joined to a predetermined number of information bits H03M13/2906 takes precedence
• CPC Class Codes: H04L 1/004 by using forward error control H04L1/0618 takes precedence; coding, decoding or code conversion, for error detection or correction H03M13/00
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/752 adapting media to network capabilities
• CPC Class Codes: H04L 65/756 adapting media to device capabilities
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04L 65/764 at the destination reformatting of additional data in video clients H04N21/4355
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04N 21/23106 involving caching operations prefetching while addressing of a memory level in which the access to the desired data or data block requires associative addressing means within memory systems or architectures G06F12/0862; caching at an intermediate stage in a data network H04L67/568
• CPC Class Codes: H04N 21/23406 involving management of server-side video buffer
• CPC Class Codes: H04N 21/234327 by decomposing into layers, e.g. base layer and one or more enhancement layers
• CPC Class Codes: H04N 21/2353 specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata
• CPC Class Codes: H04N 21/2393 involving handling client requests scheduling or organising the servicing of application requests H04L67/60
• CPC Class Codes: H04N 21/25808 Management of client data terminal profiles in network data switching protocols H04L67/303
• CPC Class Codes: H04N 21/44004 involving video buffer management, e.g. video decoder buffer or video display buffer
• CPC Class Codes: H04N 21/44209 Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896
• CPC Class Codes: H04N 21/4621 Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen arrangements for using the results of monitoring on user's side in broadcast systems H04H60/65; flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 21/8455 involving pointers to the content, e.g. pointers to the I-frames of the video stream
• CPC Class Codes: H04N 21/8456 : by decomposing the content in the time domain, e.g. in time segments

Abstract

A block-request streaming system provides for improvements in the user experience and bandwidth efficiency of such systems, typically using an ingestion system that generates data in a form to be served by a conventional file server (HTTP, FTP, or the like), wherein the ingestion system intakes content and prepares it as files or data elements to be served by the file server, which might or might not include a cache. A client device can be adapted to take advantage of the ingestion process as well as including improvements that make for a better presentation independent of the ingestion process. In the block-request streaming system, the an ingestion system generates data according to erasure codes and the client device, through various selection and timing of requests for media data and redundant data, can efficiently decode media to provide for presentations.

Claims

1. A method of requesting media data at a client device using a scalable segment map, comprising:

receiving the scalable segment map at the client device, the scalable segment map including metadata with one or more byte ranges corresponding to locations of layers of a media segment, wherein the media segment includes one or more blocks and is associated with a scalable video coded stream including multiple layers that are combinable to form one or more representations;

determining, by the client device, one or more layers of a block of the media segment to request;

generating, by the client device, a request for the one or more layers of the block, the request including a byte range corresponding to a location of the one or more layers, wherein the byte range is determined from the metadata included in the segment map;

transmitting, by the client device, the request for the one or more layers of the block;

receiving, at the client device, the one or more layers of the block of media data; and

generating, by the client device, a media representation using the one or more layers of the block.

2. The method of claim 1, wherein a full block of the one or more blocks includes layers combinable to form a first representation, and wherein a partial block of the one or more blocks has fewer layers than the full block that are combinable to form a second representation.

3. The method of claim 1, further comprising determining the one or more layers of the media segment to request based on channel quality conditions detected by the client device.

4. The method of claim 1, further comprising receiving a media presentation description including the scalable segment map.

5. The method of claim 1, further comprising receiving the media segment, wherein the media segment includes the scalable segment map.

6. The method of claim 1, wherein the metadata includes byte ranges corresponding to a location of a first layer and a location of a second layer, wherein received media data associated with the first layer is stored in a first buffer of the client device and received media data associated with the second layer is stored in a second buffer of the client device, the second buffer being a separate buffer from the first buffer.

7. The method of claim 6, wherein determining a block of media data to request includes determining an occupancy of at least one of the first buffer and the second buffer.

8. The method of claim 7, wherein determining a block of media data to request includes determining if the occupancy of the first buffer exceeds a first threshold and determining if the occupancy of the second buffer exceeds a second threshold, the first threshold being different from the second threshold.

9. The method of claim 1, wherein the layers of the media segment are ordered in ranges of increasing quality.

10. A method of providing media data, comprising:

providing, by a media ingestion system, a scalable segment map to a client device, the scalable segment map including metadata with one or more byte ranges corresponding to locations of layers of a media segment, wherein the media segment includes one or more blocks and is associated with a scalable video coded stream including multiple layers that are combinable to form a representation, wherein a subset of the multiple layers forms a first version of the representation and a larger subset of the multiple layers forms a second version of the representation having a higher quality than the first version, and wherein the scalable segment map allows the client device to generate a request for one or more layers of a block of the media segment;

receiving the request for the one or more layers of the block; and

providing the one or more layers of the block to the client device.

11. The method of claim 10, further comprising providing a media presentation description including the scalable segment map.

12. The method of claim 10, further comprising providing the media segment to the client device, wherein the media segment includes the scalable segment map.

13. The method of claim 10, wherein the layers of the media segment are ordered in ranges of increasing quality.

14. A client device configured to request media data using a scalable segment map, comprising:

a memory configured to store media data;

a receiver configured to receive the scalable segment map, the scalable segment map including metadata with one or more byte ranges corresponding to locations of layers of a media segment, wherein the media segment includes one or more blocks and is associated with a scalable video coded stream including multiple layers that are combinable to form one or more representations; and

a processor configured to;

determine one or more layers of a block of the media segment to request; and

generate a request for the one or more layers of the block, the request including a byte range corresponding to a location of the one or more layers, wherein the byte range is determined from the metadata included in the segment map;

wherein the receiver is further configured to receive the one or more layers of the block of media data; and

wherein the processor is further configured to generate a media representation using the one or more layers of the block.

15. The client device of claim 14, wherein a full block of the one or more blocks includes layers combinable to form a first representation, and wherein a partial block of the one or more blocks has fewer layers than the full block that are combinable to form a second representation.

16. The client device of claim 14, wherein the processor is further configured to determine the one or more layers of the media segment to request based on channel quality conditions detected by the client device.

17. The client device of claim 14, wherein the receiver is further configured to receive a media presentation description including the scalable segment map.

18. The client device of claim 14, wherein the receiver is further configured to receive the media segment, wherein the media segment includes the scalable segment map.

19. The client device of claim 14, further comprising:

a first buffer; and

a second buffer, wherein the second buffer is a separate buffer from the first buffer, wherein the metadata includes byte ranges corresponding to a location of a first layer and a location of a second layer, wherein received media data associated with the first layer is stored in the first buffer, and wherein received media data associated with the second layer is stored in the second buffer.

20. A server configured to provide media data, comprising:

a transmitter configured to provide a scalable segment map to a client device, the scalable segment map including metadata with one or more byte ranges corresponding to locations of layers of a media segment, wherein the media segment includes one or more blocks and is associated with a scalable video coded stream including multiple layers that are combinable to form a representation, wherein a subset of the multiple layers forms a first version of the representation and a larger subset of the multiple layers forms a second version of the representation having a higher quality than the first version, and wherein the scalable segment map allows the client device to generate a request for one or more layers of a block of the media segment; and

a receiver configured to receive the request for the one or more layers of the block;

wherein the transmitter is further configured to provide the one or more layers of the block to the client device.