Enhanced block-request streaming using cooperative parallel HTTP and forward error correction

• US 9,209,934 B2
• Filed: 09/21/2010
• Issued: 12/08/2015
• Est. Priority Date: 06/09/2006
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Luby, Michael G., Wang, Bin, Watson, Mark, Vicisano, Lorenzo, Pakzad, Payam
• Primary Examiner(s): LAMARRE, GUY J
• Application Number:
  US12/887,495
• Publication Number:
  US 20110239078A1
• Time in Patent Office: 1,904 Days
• Field of Search: 714/776, 714/E11.032, 714/752, 709/219
• US Class Current: 1/1
• CPC Class Codes: H03M 13/05 using block codes, i.e. a predetermined number of check bits joined to a predetermined number of information bits H03M13/2906 takes precedence
• CPC Class Codes: H04L 1/004 by using forward error control H04L1/0618 takes precedence; coding, decoding or code conversion, for error detection or correction H03M13/00
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/752 adapting media to network capabilities
• CPC Class Codes: H04L 65/756 adapting media to device capabilities
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04L 65/764 at the destination reformatting of additional data in video clients H04N21/4355
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04N 21/23106 involving caching operations prefetching while addressing of a memory level in which the access to the desired data or data block requires associative addressing means within memory systems or architectures G06F12/0862; caching at an intermediate stage in a data network H04L67/568
• CPC Class Codes: H04N 21/23406 involving management of server-side video buffer
• CPC Class Codes: H04N 21/234327 by decomposing into layers, e.g. base layer and one or more enhancement layers
• CPC Class Codes: H04N 21/2353 specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata
• CPC Class Codes: H04N 21/2393 involving handling client requests scheduling or organising the servicing of application requests H04L67/60
• CPC Class Codes: H04N 21/25808 Management of client data terminal profiles in network data switching protocols H04L67/303
• CPC Class Codes: H04N 21/44004 involving video buffer management, e.g. video decoder buffer or video display buffer
• CPC Class Codes: H04N 21/44209 Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896
• CPC Class Codes: H04N 21/4621 Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen arrangements for using the results of monitoring on user's side in broadcast systems H04H60/65; flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 21/8455 involving pointers to the content, e.g. pointers to the I-frames of the video stream
• CPC Class Codes: H04N 21/8456 : by decomposing the content in the time domain, e.g. in time segments

Abstract

A block-request streaming system provides for improvements in the user experience and bandwidth efficiency of such systems, typically using an ingestion system that generates data in a form to be served by a conventional file server (HTTP, FTP, or the like), wherein the ingestion system intakes content and prepares it as files or data elements to be served by the file server, which might or might not include a cache. A client device can be adapted to take advantage of the ingestion process as well as including improvements that make for a better presentation independent of the ingestion process. In the block-request streaming system, the an ingestion system generates data according to erasure codes and the client device, through various selection and timing of requests for media data and redundant data, can efficiently decode media to provide for presentations.

Claims

1. A computer-implemented method, comprising:

constructing, at a media ingestion system, media blocks and forward error correction (FEC) blocks, wherein the FEC blocks correspond to data in the media blocks;

determining a file structure, using the media ingestion system, for naming one or more media files containing the media blocks and one or more FEC files containing the FEC blocks according to a derivable pattern derivable at a client device;

receiving a first request from the client device for a first media file that is usable by the client device to provide a media presentation;

responding to the first request by initiating transmission of the first media file;

receiving a second request from the client device for a second media file that is usable by the client device to provide additional portions of the media presentation;

responding to the second request by initiating transmission of the second media file prior to completion of the response to the first request;

receiving a third request from the client device for an FEC file that is usable by the client device to recover portions of either the first media file or the second media file not yet received by the client device; and

responding to the third request by initiating transmission of the FEC file prior to completion of the response to the first request and prior to completion of the response to the second request.

2. The method of claim 1, further comprising:

receiving a request from one or more clients for media files; and

responding to the request from the one or more clients by sending the requested media files.

3. The method of claim 2, further comprising:

receiving at least one additional request for a media file prior to completing a response to a previous request for a media file; and

initiating a response to the at least one additional request prior to completing the response to the previous request.

4. The method of claim 1, wherein media blocks are stored as pluralities of media files and further comprising configuring the media ingestion system to provide a media block using a plurality of requests for the pluralities of media files.

5. The method of claim 1, further comprising:

responding to more than one request in parallel;

determining a bandwidth allocation among a plurality of responses to the parallel requests;

determining playout times associated with the parallel requests; and

adjusting the bandwidth allocation to reduce a probability of a stall, wherein a stall occurs when a playout time is reached before media that is associated with that playout time is available for playout.

6. The method of claim 1, wherein a plurality of requests for media files comprises a plurality of requests for byte ranges from one or more files.

7. The method of claim 1, wherein a plurality of requests for media files comprises a plurality of requests for different layers of a presentation, wherein one representation of a media presentation can be generated from one layer and another representation of a media presentation is generated from more than one layer.

8. The method of claim 1, further comprising processing the media files to generate the FEC files such that a number of FEC symbols used for a fragment of the media files is determined by a target repair data ratio, a symbol size, a byte offset of a prior media fragment and a byte offset of a current media fragment.

9. An apparatus, comprising:

a processor configured to;

construct media blocks and forward error correction (FEC) blocks, wherein the FEC blocks correspond to data in the media blocks; and

determine a file structure for naming one or more media files containing the media blocks and one or more FEC files containing the FEC blocks according to a derivable pattern derivable at a client device; and

a receiver configured to receive a first request from the client device for a first media file that is usable by the client device to provide a media presentation;

wherein the processor is configured to respond to the first request by initiating transmission of the first media file;

wherein the receiver is configured to receive a second request from the client device for a second media file that is usable by the client device to provide additional portions of the media presentation;

wherein the processor is configured to respond to the second request by initiating transmission of the second media file prior to completion of the response to the first request;

wherein the receiver is configured to receive a third request from the client device for an FEC file that is usable by the client device to recover portions of either the first media file or the second media file not yet received by the client device; and

wherein the processor is configured to respond to the third request by initiating transmission of the FEC file prior to completion of the response to the first request and prior to completion of the response to the second request.

10. The apparatus of claim 9, wherein:

the receiver is configured to receive a request from one or more clients for media files; and

the processor is configured to respond to the request from the one or more clients by sending the requested media files.

11. The apparatus of claim 10, wherein:

the receiver is configured to receive at least one additional request for a media file prior to completing a response to a previous request for a media file; and

the processor is configured to initiate a response to the at least one additional request prior to completing the response to the previous request.

12. The apparatus of claim 9, wherein media blocks are stored as pluralities of media files and wherein the processor is configured to configure a media ingestion system to provide a media block using a plurality of requests for the pluralities of media files.

13. The apparatus of claim 9, wherein the processor is configured to:

respond to more than one request in parallel;

determine a bandwidth allocation among a plurality of responses to the parallel requests;

determine playout times associated with the parallel requests; and

adjust the bandwidth allocation to reduce a probability of a stall, wherein a stall occurs when a playout time is reached before media that is associated with that playout time is available for playout.

14. The apparatus of claim 9, wherein a plurality of requests for media files comprises a plurality of requests for byte ranges from one or more files.

15. The apparatus of claim 9, wherein a plurality of requests for media files comprises a plurality of requests for different layers of a presentation, wherein one representation of a media presentation is generated from one layer and another representation of a media presentation is generated from more than one layer.

16. The apparatus of claim 9, wherein the processor is configured to process the media files to generate the FEC files such that a number of FEC symbols used for a fragment of the media files is determined by a target repair data ratio, a symbol size, a byte offset of a prior media fragment and a byte offset of a current media fragment.

17. One or more computer-readable media storing computer-executable instructions that, when executed, cause one or more computing devices to:

construct media blocks and forward error correction (FEC) blocks, wherein the FEC blocks correspond to data in the media blocks;

determine a file structure for naming one or more media files containing the media blocks and one or more FEC files containing the FEC blocks according to a derivable pattern derivable at the client device;

receive a first request from the client device for a first media file that is usable by the client device to provide a media presentation;

respond to the first request by initiating transmission of the first media file;

receive a second request from the client device for a second media file that is usable by the client device to provide additional portions of the media presentation;

respond to the second request by initiating transmission of the second media file prior to completion of the response to the first request;

receive a third request from the client device for an FEC file that is usable by the client device to recover portions of either the first media file or the second media file not yet received by the client device; and

respond to the third request by initiating transmission of the FEC file prior to completion of the response to the first request and prior to completion of the response to the second request.