Trick modes for network streaming of coded multimedia data

• US 9,319,448 B2
• Filed: 08/08/2011
• Issued: 04/19/2016
• Est. Priority Date: 08/10/2010
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Chen, Ying, Stockhammer, Thomas, Watson, Mark
• Primary Examiner(s): Gee, Jason K.
• Assistant Examiner(s): BUCKNOR, OLANREWAJU J
• Application Number:
  US13/205,565
• Publication Number:
  US 20120042089A1
• Time in Patent Office: 1,716 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/752 adapting media to network capabilities
• CPC Class Codes: H04L 65/756 adapting media to device capabilities
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04L 65/764 at the destination reformatting of additional data in video clients H04N21/4355
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04N 21/8455 involving pointers to the content, e.g. pointers to the I-frames of the video stream
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments

Abstract

In one example, a device for retrieving multimedia data, the device comprising one or more processors configured to analyze information of a manifest file for multimedia content, wherein the information of the manifest file indicates that at least one representation of the multimedia content includes a temporal sub-sequence, determine one or more locations of data for the temporal sub-sequence, and submit one or more requests for the data for the temporal sub-sequence.

Claims

1. A method of retrieving multimedia data, the method comprising:

analyzing, by a client device, information of a manifest file for multimedia content, wherein the information of the manifest file indicates that at least one representation of the multimedia content includes a temporal sub-sequence, wherein the representation includes a bitstream having a plurality of segments formed as individually retrievable files having respective uniform resource locators (URLs), wherein each of the segments of the representation includes a respective random access point (RAP) picture and a respective plurality of non-RAP pictures, wherein the temporal sub-sequence is formed by at least some of the RAP pictures of the segments and none of the non-RAP pictures, and wherein the manifest file is separate from each of the plurality of segments;

determining, by the client device, byte ranges of the segments corresponding to locations of the RAP pictures for the temporal sub-sequence in the segments, wherein the byte ranges are each defined by a respective starting byte corresponding to a beginning of the RAP picture of the corresponding segment and an ending byte corresponding to an end of the RAP picture of the corresponding segment; and

submitting, by the client device, requests for the data for the temporal sub-sequence, wherein the requests comprise partial GET requests that specify, for each of the segments, the URL of the segment, the starting byte of the RAP picture of the segment, and the ending byte of the RAP picture of the segment.

2. The method of claim 1, further comprising presenting the data of the temporal sub-sequence in a trick mode for the representation.

3. The method of claim 1, wherein determining the byte ranges of the segments comprises determining the byte ranges of the segments for the temporal sub-sequence from the manifest file.

4. The method of claim 1, wherein determining the byte ranges of the segments comprises, for each of the segments:

retrieving data for a portion of the segment, wherein the portion of the segment includes data indicative of the byte range of the RAP picture of the segment ranges; and

analyzing the data for the retrieved portion of the segment to determine the byte range of the RAP picture of the segment.

5. The method of claim 4, wherein the data for the portion of the segment defines a continuous byte sequence of the segment that includes the RAP picture of the segment, and wherein submitting the requests comprises submitting a single request for the segment that specifies the byte range defined by the continuous byte sequence.

6. The method of claim 4, wherein the portion of the segment comprises a sub-fragment index box of the segment.

7. The method of claim 4, wherein retrieving the data for the portion of the segment comprises:

determining a starting byte of the portion of the segment and an ending byte of the portion of the segment from data of the manifest file; and

sending a partial GET request that specifies the starting byte of the portion of the segment, the ending byte of the portion of the segment, and the URL of the segment.

8. The method of claim 1, wherein the data for the temporal sub-sequence comprises one or more instantaneous decoder refresh (IDR) pictures of the representation.

9. A device for retrieving multimedia data, the device comprising one or more processors configured to:

analyze information of a manifest file for multimedia content, wherein the information of the manifest file indicates that at least one representation of the multimedia content includes a temporal sub-sequence, wherein the representation includes a bitstream having a plurality of segments formed as individually retrievable files having respective uniform resource locators (URLs), wherein each of the segments of the representation includes a respective random access point (RAP) picture and a respective plurality of non-RAP pictures, wherein the temporal sub-sequence is formed by at least some of the RAP pictures of the segments and none of the non-RAP pictures, and wherein the manifest file is separate from each of the plurality of segments,

determine byte ranges of the segments corresponding to locations of the RAP pictures for the temporal sub-sequence in the segments, wherein the byte ranges are each defined by a respective starting byte corresponding to a beginning of the RAP picture of the corresponding segment and an ending byte corresponding to an end of the RAP picture of the corresponding segment, and

submit requests for the data for the temporal sub-sequence, wherein the requests comprise partial GET requests that specify, for each of the segments, the URL of the segment, the starting byte of the RAP picture of the segment, and the ending byte of the RAP picture of the segment.

10. The device of claim 9, wherein the one or more processors are configured to determine the byte ranges of the segments for the temporal sub-sequence from the manifest file.

11. The device of claim 9, wherein to determine the byte ranges of the segments, the one or more processors are configured to, for each of the segments:

retrieve data for a portion of the segment, wherein the portion of the segment includes data indicative of the byte range of the RAP picture of the segment, and

analyze the data for the retrieved portion of the segment to determine the byte range of the RAP picture of the segment.

12. The device of claim 11, wherein the data for the portion of the segment comprises a sub-fragment index box of the segment, wherein the data for the portion of the segment defines a continuous byte sequence of the segment that includes the RAP picture of the segment, and wherein the device is configured to submit a single request for the segment that specifies the byte range defined by the continuous byte sequence.

13. The device of claim 11, wherein to retrieve the data for the portion of the segment, the one or more processors are configured to:

determine a starting byte of the portion of the segment and an ending byte of the portion of the segment from data of the manifest file, and

send a partial GET request that specifies the starting byte of the portion of the segment, the ending byte of the portion of the segment, and the URL of the segment.

14. The device of claim 9, wherein the device comprises at least one of:

an integrated circuit;

a microprocessor; and

a wireless communication device that includes the one or more processors.

15. A device for retrieving multimedia data, the device comprising:

means for analyzing information of a manifest file for multimedia content, wherein the information of the manifest file indicates that at least one representation of the multimedia content includes a temporal sub-sequence, wherein the representation includes a bitstream having a plurality of segments formed as individually retrievable files having respective uniform resource locators (URLs), wherein each of the segments of the representation includes a respective random access point (RAP) picture and a respective plurality of non-RAP pictures, wherein the temporal sub-sequence is formed by at least some of the RAP pictures of the segments and none of the non-RAP pictures, and wherein the manifest file is separate from each of the plurality of segments;

means for determining byte ranges of the segments corresponding to locations of the RAP pictures for the temporal sub-sequence in the segments, wherein the byte ranges are each defined by a respective starting byte corresponding to a beginning of the RAP picture of the corresponding segment and an ending byte corresponding to an end of the RAP picture of the corresponding segment; and

means for submitting requests for the data for the temporal sub-sequence, wherein the requests comprise partial GET requests that specify, for each of the segments, the URL of the segment, the starting byte of the RAP picture of the segment, and the ending byte of the RAP picture of the segment.

16. The device of claim 15, wherein the means for determining the byte ranges of the segments comprises means for determining the byte ranges of the segments for the temporal sub-sequence from the manifest file.

17. The device of claim 15, wherein the means for determining the byte ranges of the segments comprises:

means for retrieving, for each of the segments, data for a portion of the segment, wherein the portion of the segment includes data indicative of the byte range of the RAP picture of the segment; and

means for analyzing, for each of the segments, the data for the retrieved portion of the segment to determine the byte range of the RAP picture of the segment.

18. The device of claim 17, wherein the portion of the segment comprises a sub-fragment index box of the segment, wherein the data for the portion of the segment defines a continuous byte sequence of the segment that includes the RAP picture of the segment, and wherein the means for submitting the requests comprise means for submitting a single request for the segment that specifies the byte range defined by the continuous byte sequence.

19. The device of claim 17, wherein the means for retrieving the data for the portion of the segment comprises:

means for determining a starting byte of the portion of the segment and an ending byte of the portion of the segment from data of the manifest file; and

means for sending a partial GET request that specifies the starting byte of the portion of the segment, the ending byte of the portion of the segment, and the URL of the segment.

20. A computer program product comprising a non-transitory computer-readable medium having stored thereon instructions that, when executed, cause a processor of a device for retrieving multimedia data to:

analyze information of a manifest file for multimedia content, wherein the information of the manifest file indicates that at least one representation of the multimedia content includes a temporal sub-sequence, wherein the representation includes a bitstream having a plurality of segments formed as individually retrievable files having respective uniform resource locators (URLs), wherein each of the segments of the representation includes a respective random access point (RAP) picture and a respective plurality of non-RAP pictures, wherein the temporal sub-sequence is formed by at least some of the RAP pictures of the segments and none of the non-RAP pictures, and wherein the manifest file is separate from each of the plurality of segments;

determine byte ranges of the segments corresponding to locations of the RAP pictures for the temporal sub-sequence in the segments, wherein the byte ranges are each defined by a respective starting byte corresponding to a beginning of the RAP picture of the corresponding segment and an ending byte corresponding to an end of the RAP picture of the corresponding segment; and

submit requests for the data for the temporal sub-sequence, wherein the requests comprise partial GET requests that specify, for each of the segments, the URL of the segment, the starting byte of the RAP picture of the segment, and the ending byte of the RAP picture of the segment.

21. The computer program product of claim 20, wherein the instructions cause the processor to present the temporal sub-sequence of the representation in a trick mode for the representation.

22. The computer program product of claim 20, wherein the instructions that cause the processor to determine the locations of the data comprise instructions that cause the processor to determine the byte ranges of the segments for the temporal sub-sequence from the manifest file.

23. The computer program product of claim 20, wherein the instructions that cause the processor to determine the byte ranges of the segments comprise instructions that cause the processor to, for each of the segments:

retrieve data for a portion of the segment, wherein the portion of the segment includes data indicative of the byte range of the RAP picture of the segment, and

analyze the data for the retrieved portion of the segment to determine the byte range of the RAP picture of the segment.

24. The computer program product of claim 23, wherein the data for the portion of the segment comprises a sub-fragment index box of the segment, wherein the data for the portion of the segment defines a continuous byte sequence of the segment that includes the RAP picture of the segment, and wherein the instructions that cause the processor to submit the requests comprise instructions that cause the processor to submit a single request for the segment that specifies the byte range defined by the continuous byte sequence.

25. The computer program product of claim 23, wherein the instructions that cause the processor to retrieve the data for the portion of the representation comprise instructions that cause the processor to:

determine a starting byte of the portion of the segment and an ending byte of the portion of the representation segment from data of the manifest file, and

send a partial GET request that specifies the starting byte of the portion of the segment, the ending byte of the portion of the segment, and the URL of the segment.

26. A method of sending information for multimedia data, the method comprising:

obtaining data for at least one representation of multimedia content that includes a temporal sub-sequence, wherein the representation includes a bitstream having a plurality of segments formed as individually retrievable files having respective uniform resource locators (URLs), wherein each of the segments of the representation includes a respective random access point (RAP) picture and a respective plurality of non-RAP pictures, and wherein the temporal sub-sequence is formed by at least some of the RAP pictures of the segments and none of the non-RAP pictures;

obtaining data for a manifest file for the multimedia content, wherein information of the manifest file indicates that the at least one representation of the multimedia content includes the temporal sub-sequence, and wherein the manifest file is separate from each of the plurality of segments;

sending at least a portion of the manifest file to a client device; and

sending data indicative of byte ranges corresponding to locations of the RAP pictures for the temporal sub-sequence in the segments to the client device, wherein the byte ranges are each defined by a respective starting byte corresponding to a beginning of the RAP picture of the corresponding segment and an ending byte corresponding to an end of the RAP picture of the corresponding segment.

27. The method of claim 26, wherein the manifest file includes the data indicative of the byte ranges corresponding to the locations of the RAP pictures for the temporal sub-sequence.

28. The method of claim 26, wherein a portion of each of the segments includes the data indicative of the byte ranges of the RAP picture for the temporal sub-sequence, the method further comprising, for each of the segments:

receiving a request for the portion of the segment from the client device; and

sending the portion of the segment to the client device in response to the request.

29. The method of claim 28, wherein the portion of the segment comprises a sub-fragment index box of the segment, and wherein the data for the portion of the segment defines a continuous byte sequence of the segment that includes the RAP picture of the segment.

30. The method of claim 28, wherein the manifest file indicates a byte range for the portion of the segment, and wherein the request comprises a partial GET request that specifies the byte range for the portion of the segment and the URL of the segment.

31. A device for sending information for multimedia data, the device comprising one or more processors configured to:

obtain data for at least one representation of multimedia content that includes a temporal sub-sequence, wherein the representation includes a bitstream having a plurality of segments formed as individually retrievable files having respective uniform resource locators (URLs), wherein each of the segments of the representation includes a respective random access point (RAP) picture and a respective plurality of non-RAP pictures, and wherein the temporal sub-sequence is formed by at least some of the RAP pictures of the segments and none of the non-RAP pictures,

obtain data for a manifest file for the multimedia content, wherein information of the manifest file indicates that the at least one representation of the multimedia content includes the temporal sub-sequence, and wherein the manifest file is separate from each of the plurality of segments,

send at least a portion of the manifest file to a client device, and

send data indicative of byte ranges corresponding to locations of the RAP pictures for the temporal sub-sequence in the segments to the client device, wherein the byte ranges are each defined by a respective starting byte corresponding to a beginning of the RAP picture of the corresponding segment and an ending byte corresponding to an end of the RAP picture of the corresponding segment.

32. The device of claim 31, wherein the manifest file includes the data indicative of the byte ranges corresponding to the locations of the RAP pictures for the temporal sub-sequence.

33. The device of claim 31, wherein a portion of each of the segments includes the data indicative of the byte ranges of the RAP picture for the temporal sub-sequence, and wherein the one or more processors are configured to, for each of the segments:

receive a request for the portion of the segment from the client device, and

send the portion of the segment to the client device in response to the request.

34. The device of claim 33, wherein the portion of the segment comprises a sub-fragment index box of the segment, and wherein the data for the portion of the segment defines a continuous byte sequence of the segment that includes the RAP picture of the segment.

35. The device of claim 33, wherein the manifest file indicates a byte range for the portion of the segment, and wherein the request comprises a partial GET request that specifies the byte range for the portion of the segment and the URL of the segment.

36. The device of claim 31, wherein the device comprises at least one of:

an integrated circuit;

a microprocessor; and

a wireless communication device that includes the one or more processors.

37. A device for sending information for multimedia data, the device comprising:

means for obtaining data for at least one representation of multimedia content that includes a temporal sub-sequence, wherein the representation includes a bitstream having a plurality of segments formed as individually retrievable files having respective uniform resource locators (URLs), wherein each of the segments of the representation includes a respective random access point (RAP) picture and a respective plurality of non-RAP pictures, and wherein the temporal sub-sequence is formed by at least some of the RAP pictures of the segments and none of the non-RAP pictures;

means for obtaining data for a manifest file for the multimedia content, wherein information of the manifest file indicates that the at least one representation of the multimedia content includes the temporal sub-sequence, and wherein the manifest file is separate from each of the plurality of segments;

means for sending at least a portion of the manifest file to a client device; and

means for sending data indicative of byte ranges corresponding to locations of the RAP pictures for the temporal sub-sequence in the segments to the client device, wherein the byte ranges are each defined by a respective starting byte corresponding to a beginning of the RAP picture of the corresponding segment and an ending byte corresponding to an end of the RAP picture of the corresponding segment.

38. The device of claim 37, wherein the manifest file includes the data indicative of the byte ranges corresponding to the locations of the RAP pictures for the temporal sub-sequence.

39. The device of claim 37, wherein a portion of each of the segments includes the data indicative of the byte ranges of the RAP picture for the temporal sub-sequence, further comprising:

means for receiving, for each of the segments, a request for the portion of the segment from the client device; and

means for sending, for each of the segments, the portion of the segment to the client device in response to the request.

40. The device of claim 39, wherein the portion of the segment comprises a sub-fragment index box of the segment, and wherein the data for the portion of the segment defines a continuous byte sequence of the segment that includes the RAP picture of the segment.

41. The device of claim 39, wherein the manifest file indicates a byte range for the portion of the segment, and wherein the request comprises a partial GET request that specifies the byte range for the portion of the segment and the URL of the segment.

42. A computer program product comprising a non-transitory computer-readable medium having stored thereon instructions that, when executed, cause a processor of a device for sending information for multimedia data to:

obtain data for at least one representation of multimedia content that includes a temporal sub-sequence, wherein the representation includes a bitstream having a plurality of segments formed as individually retrievable files having respective uniform resource locators (URLs), wherein each of the segments of the representation includes a respective random access point (RAP) picture and a respective plurality of non-RAP pictures, and wherein the temporal sub-sequence is formed by at least some of the RAP pictures of the segments and none of the non-RAP pictures;

obtain data for a manifest file for the multimedia content, wherein information of the manifest file indicates that the at least one representation of the multimedia content includes the temporal sub-sequence, and wherein the manifest file is separate from each of the plurality of segments;

send at least a portion of the manifest file to a client device; and

send data indicative of byte ranges corresponding to locations of the RAP pictures for the temporal sub-sequence in the segments to the client device, wherein the byte ranges are each defined by a respective starting byte corresponding to a beginning of the RAP picture of the corresponding segment and an ending byte corresponding to an end of the RAP picture of the corresponding segment.

43. The computer program product of claim 42, wherein the manifest file includes the data indicative of the byte ranges corresponding to the locations of the RAP pictures for the temporal sub-sequence.

44. The computer program product of claim 42, wherein a portion of each of the segments includes the data indicative of the byte ranges of the RAP picture for the temporal sub-sequence, further comprising instructions that cause the processor to, for each of the segments:

receive a request for the portion of the segment from the client device; and

send the portion of the segment to the client device in response to the request.

45. The computer program product of claim 44, wherein the portion of the segment comprises a sub-fragment index box of the segment, and wherein the data for the portion of the segment defines a continuous byte sequence of the segment that includes the RAP picture of the segment.

46. The computer program product of claim 44, wherein the manifest file indicates a byte range for the portion of the segment, and wherein the request comprises a partial GET request that specifies the byte range for the portion of the segment and the URL of the segment.