Representation groups for network streaming of coded multimedia data

• US 9,456,015 B2
• Filed: 08/08/2011
• Issued: 09/27/2016
• Est. Priority Date: 08/10/2010
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Chen, Ying, Stockhammer, Thomas, Watson, Mark
• Primary Examiner(s): MIRZA, ADNAN M
• Application Number:
  US13/205,559
• Publication Number:
  US 20120042050A1
• Time in Patent Office: 1,877 Days
• Field of Search: 709/218, 709/219
• US Class Current: 1/1
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/752 adapting media to network capabilities
• CPC Class Codes: H04L 65/756 adapting media to device capabilities
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04L 65/764 at the destination reformatting of additional data in video clients H04N21/4355
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04N 21/8455 involving pointers to the content, e.g. pointers to the I-frames of the video stream
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments

Abstract

In one example, a device for receiving information for multimedia data, the device comprising one or more processors configured to analyze at least a portion of a manifest file for multimedia content, wherein the portion of the manifest file includes information indicative of sets of representations of the multimedia content and information indicative of common characteristics for each of the sets of representations, select one of the sets of representations based on the common characteristics for the one of the sets of representations, select one of the representations of the selected one of the sets of representations based on one or more coding characteristics of the one of the representations of the one of the sets, and generate a request for data of the one of the representations based on the selection.

Claims

1. A method of retrieving multimedia data, the method comprising:

analyzing, by a client device, at least a first portion of a manifest file for multimedia content, wherein the first portion of the manifest file includes information indicative of sets of representations of the multimedia content, information indicative of common characteristics for each of the sets of representations, and information indicative of a location of a second portion of the manifest file, wherein analyzing the at least first portion comprises determining the location of the second portion of the manifest file, wherein the manifest file is separate from the representations, and wherein the representations included in each of the sets of representations comprise switchable representations that are alternatives to each other and that each represent the same multimedia content;

selecting, by the client device, one of the sets of representations based on the common characteristics for the one of the sets of representations such that the common characteristics correspond to configuration data of the client device;

selecting, by the client device, one of the representations of the selected one of the sets of representations based on one or more coding characteristics of the one of the representations of the one of the sets;

generating, by the client device, a request for a first set of multimedia data of the one of the representations based on the selection;

retrieving, by the client device, at least the second portion of the manifest file from the location determined from the first portion of the manifest file;

analyzing, by the client device, the at least second portion of the manifest file to determine a location of a second set of multimedia data; and

retrieving, by the client device, at least a portion of the second set of multimedia data from the location determined from the second portion of the manifest file.

2. The method of claim 1, wherein the multimedia content corresponding to the manifest file comprises requested multimedia content, the method further comprising:

prior to sending the request for the first set of multimedia data, sending user information including at least one of a user identifier, user preferences for advertisements, and user demographic information;

wherein analyzing the at least first portion of the manifest file comprises selecting a representation of a remote period comprising targeted advertisement multimedia content from the manifest file received in response to the user information, wherein the representation of the remote period comprises the second set of multimedia data;

wherein retrieving the at least portion of the second set of multimedia data comprises sending a request for the second set of multimedia data of the selected representation of the targeted advertisement multimedia content; and

sending the request for the first set of multimedia data of the one of the representations of the requested multimedia content only after sending the request for the second set of multimedia data of the selected representation of the targeted advertisement multimedia content.

3. The method of claim 1, wherein the second portion of the manifest file comprises a different portion of the manifest file than the first portion, wherein the first set of multimedia data and the second set of multimedia data comprise the same set of multimedia data, and wherein analyzing the at least second portion of the manifest file comprises determining the location of the set of multimedia data of the one of the representations from the second portion of the manifest file.

4. The method of claim 3, wherein the second portion of the manifest file includes information indicative of coding characteristic for each of the representations in the one of the sets of representations, and wherein selecting the one of the representations comprises analyzing the second portion of the manifest file to determine the coding characteristics of the representations in the one of the sets.

5. The method of claim 3, further comprising:

determining bitrates for the representations in the one of the sets from information included in the second portion of the manifest file;

wherein selecting the one of the representations comprises;

determining an available amount of network bandwidth; and

selecting the one of the representations based on a comparison of the bitrates for the representations in the one of the sets and the available amount of network bandwidth.

6. The method of claim 5, further comprising:

after sending the request for the set of multimedia data of the one of the representations, determining that the available amount of network bandwidth has changed;

selecting a second, different representation of the one of the sets of representations based on a comparison of the bitrates for the representations in the one of the sets and the changed amount of network bandwidth; and

sending a second request for data of the selected second representation of the one of the sets of representations, wherein the second request specifies data of a temporal period of the multimedia content subsequent to a temporal period of the multimedia content for the data of the first request.

7. The method of claim 1, wherein the common characteristics for the set of representations include information indicative of a language for at least one of text in the set of representations and a language for audio data in the set of representations, and wherein selecting the one of the sets of representations comprises selecting the one of the sets of representations having at least one of a language for text and audio corresponding to a language indicated by the configuration data of the client device.

8. The method of claim 1, wherein the common characteristics for the set of representations include information indicative of a camera perspective of a scene for the set of representations, and wherein selecting the one of the sets of representations comprises selecting the one of the sets of representations having a camera perspective corresponding to a camera perspective indicated by the configuration data of the client device.

9. The method of claim 1, wherein the common characteristics for the set of representations include information indicative of a rating for the set of representations, wherein the rating describes content suitability for a particular audience, and wherein selecting the one of the sets of representations comprises selecting the one of the sets of representations having a rating corresponding to a rating indicated by the configuration data of the client device.

10. The method of claim 1, further comprising determining a profile to which the manifest file conforms and retrieving the manifest file based on the profile.

11. A client device for receiving information for multimedia data, the client device comprising one or more processors configured to:

analyze at least a first portion of a manifest file for multimedia content, wherein the portion of the manifest file includes information indicative of sets of representations of the multimedia content, information indicative of common characteristics for each of the sets of representations, and information indicative of the location of a second portion of the manifest file, wherein the manifest file is separate from the representations, and wherein the representations included in each of the sets of representations comprise switchable representations that are alternatives to each other and that each represent the same multimedia content,

determine a location of the second portion of the manifest file,

select one of the sets of representations based on the common characteristics for the one of the sets of representations such that the common characteristics correspond to configuration data of the client device,

select one of the representations of the selected one of the sets of representations based on one or more coding characteristics of the one of the representations of the one of the sets,

generate a request for data of the one of the representations based on the selection, retrieve at least the second portion of the manifest file from the location determined from the first portion of the manifest file,

analyze the at least second portion of the manifest file to determine a location of a second set of multimedia data, and

retrieve at least a portion of the second set of multimedia data from the location determined from the second portion of the manifest file.

12. The client device of claim 11, wherein the multimedia content corresponding to the first portion of the manifest file comprises requested multimedia content, and wherein the one or more processors are configured to, prior to sending the request for the first set of multimedia data, send user information including at least one of a user identifier, user preferences for advertisements, and user demographic information, wherein to analyze the at least first portion of the manifest file, the one or more processors are configured to select a representation of a remote period comprising targeted advertisement multimedia content from the manifest file received in response to the user information, wherein the representation of the remote period comprises the second set of multimedia data, wherein to retrieve the at least portion of the second set of multimedia data, the one or more processors are configured to send a request for the second set of multimedia data of the selected representation of the targeted advertisement multimedia content, and to send the request for the first set of multimedia data of the one of the representations of the requested multimedia content only after sending the request for the second set of multimedia data of the selected representation of the targeted advertisement multimedia content.

13. The client device of claim 11, wherein the second portion of the manifest file comprises a different portion of the manifest file than the first portion, wherein the first set of multimedia data and the second set of multimedia data comprise the same set of multimedia data, and wherein the one or more processors are configured to determine the location of the set of multimedia data of the one of the representations from the second portion of the manifest file.

14. The client device of claim 13, wherein the second portion of the manifest file includes information indicative of coding characteristic for each of the representations in the one of the sets of representations, and wherein to select the one of the representations, the one or more processors are configured to analyze the second portion of the manifest file to determine the coding characteristics of the representations in the one of the sets.

15. The client device of claim 11, wherein the common characteristics for the set of representations include at least one of information indicative of a language for at least one of text and audio in the set of representations, information indicative of a camera perspective of a scene for the set of representations, and information indicative of a rating for the set of representations, wherein the rating describes content suitability for a particular audience and wherein to select the one of the sets of representations, the one or more processors are configured to select the one of the sets of representations having at least one of a language for at least one of text and audio corresponding to a language indicated by the configuration data of the client device, a camera perspective corresponding to a camera perspective indicated by the configuration data of the client device, and a rating corresponding to a rating indicated by the configuration data of the client device.

16. The client device of claim 11, wherein the client device comprises at least one of:

an integrated circuit;

a microprocessor; and

a wireless communication device that includes the one or more processors.

17. A computer program product comprising a non-transitory computer-readable medium having stored thereon instructions that, when executed, cause a processor of a client device for retrieving multimedia data to:

analyze at least a first portion of a manifest file for multimedia content, wherein the portion of the manifest file includes information indicative of sets of representations of the multimedia content, information indicative of common characteristics for each of the sets of representations, and information indicative of a location of a second portion of the manifest file, wherein the instructions that cause the processor to analyze the at least first portion comprise instructions that cause the processor to determine the location of the second portion of the manifest file, wherein the manifest file is separate from the representations, and wherein the representations included in each of the sets of representations comprise switchable representations that are alternatives to each other and that each represent the same multimedia content;

select one of the sets of representations based on the common characteristics for the one of the sets of representations such that the common characteristics correspond to configuration data of the client device;

select one of the representations of the selected one of the sets of representations based on one or more coding characteristics of the one of the representations of the one of the sets;

generate a request for a first set of multimedia data of the one of the representations based on the selection;

retrieve at least the second portion of the manifest file from the location determined from the first portion of the manifest file;

analyze the at least second portion of the manifest file to determine a location of a second set of multimedia data; and

retrieve at least a portion of the second set of multimedia data from the location determined from the second portion of the manifest file.

18. The computer program product of claim 17, wherein the multimedia content corresponding to the first manifest file comprises requested multimedia content, further comprising instructions that cause the processor to:

prior to sending the request for the first set of multimedia data, send user information including at least one of a user identifier, user preferences for advertisements, and user demographic information;

wherein the instructions that cause the processor to analyze the at least first portion of the manifest file comprise instructions that cause the processor to select a representation of a remote period comprising targeted advertisement multimedia content from the manifest file received in response to the user information, wherein the representation of the remote period comprises the second set of multimedia data;

wherein the instructions that cause the processor to retrieve the at least portion of the second set of multimedia data comprises instructions that cause the processor to send a request for the second set of multimedia data of the selected representation of the targeted advertisement multimedia content; and

send the request for the first set of multimedia data of the one of the representations of the requested multimedia content only after sending the request for the second set of multimedia data of the selected representation of the targeted advertisement multimedia content.

19. The computer program product of claim 17, wherein the common characteristics for the set of representations include at least one of information indicative of a language for at least one of text and audio in the set of representations, information indicative of a camera perspective of a scene for the set of representations, and information indicative of a rating for the set of representations, wherein the rating describes content suitability for a particular audience, and wherein the instructions that cause the processor to select the one of the sets of representations comprise instructions that cause the processor to select the one of the sets of representations having at least one of a language for at least one of text and audio corresponding to a language indicated by the configuration data of the client device, a camera perspective corresponding to a camera perspective indicated by the configuration data of the client device, and a rating corresponding to a rating indicated by the configuration data of the client device.

38. The computer program product of claim 17, wherein the second portion of the manifest file comprises a different portion of the manifest file than the first portion, wherein the first set of multimedia data and the second set of multimedia data comprise the same set of multimedia data, and wherein the instructions that cause the processor to analyze the at least second portion of the manifest file comprises instructions that cause the processor to determine the location of the set of multimedia data of the one of the representations from the second portion of the manifest file.

20. A method of sending information for multimedia data, the method comprising:

obtaining a set of representations of multimedia content having one or more common characteristics, wherein each of the representations in the set have individual coding characteristics separate from the common characteristics;

obtaining a manifest file for the multimedia content, wherein the manifest file includes information indicative of the representations in the set, information indicative of the common characteristics for the set of representations, information indicative of the coding characteristics for each of the representations in the set, and information indicative of locations of a first set of multimedia data of at least one of the representations, and wherein the manifest file comprises a first portion and a second, different portion, wherein the first portion of the manifest file includes the information indicative of the representations in the set, the information indicative of the common characteristics, and information indicative of a location of the second portion of the manifest file, wherein the second portion includes information indicative of a location of a second set of multimedia data, wherein the manifest file is separate from the representations, and wherein the representations included in each of the sets of representations comprise switchable representations that are alternatives to each other and that each represent the same multimedia content; and

sending at least the first portion and the second portion of the manifest file to a client device.

21. The method of claim 20, wherein the common characteristics for the set of representations include information indicative of at least one of a language for at least one of text and audio in the set of representations, a camera perspective of a scene for the set of representations, and a rating for the set of representations, wherein the rating describes content suitability for a particular audience.

22. The method of claim 20, further comprising:

receiving, from the client device, a request for the second portion of the manifest file;

sending the second portion of the manifest file to the client device in response to the request;

receiving, from the client device, a request for at least a portion of the second set of multimedia data from the location indicated by the second portion of the manifest file; and

sending, in response to the request for the portion of the second set of multimedia data, the requested portion of the second set of multimedia data to the client device.

23. The method of claim 22, wherein the multimedia content corresponding to the first portion of the manifest file comprises requested multimedia content, the method further comprising:

receiving, from the client device, user information including at least one of a user identifier, user preferences for advertisements, and user demographic information; and

selecting targeted advertisement multimedia data based on the received user information, wherein the selected targeted advertisement multimedia data is associated with the second portion of the manifest file.

24. The method of claim 22, wherein the second portion of the manifest file comprises a different portion of the manifest file than the first portion, and wherein the first set of multimedia data and the second set of multimedia data comprise the same set of multimedia data.

25. The method of claim 20, further comprising:

sending, before sending the at least first and second portions of the manifest file, information indicative of a profile to which the manifest file corresponds to the client device; and

receiving a request for the manifest file from the client device after sending the information indicative of the profile.

26. A device for sending information for multimedia data, the device comprising one or more processors configured to:

obtain a set of representations of multimedia content having one or more common characteristics, wherein each of the representations in the set have individual coding characteristics separate from the common characteristics,

obtain a manifest file for the multimedia content, wherein the manifest file includes information indicative of the representations in the set, information indicative of the common characteristics for the set of representations, information indicative of the coding characteristics for each of the representations in the set, and information indicative of locations of a first set of multimedia data of at least one of the representations, and wherein the manifest file comprises a first portion and a second, different portion, wherein the first portion of the manifest file includes the information indicative of the representations in the set, the information indicative of the common characteristics, and information indicative of a location of the second portion of the manifest file, wherein the second portion includes information indicative of a location of a second set of multimedia data, wherein the manifest file is separate from the representations, and wherein the representations included in each of the sets of representations comprise switchable representations that are alternatives to each other and that each represent the same multimedia content, and

send at least the first portion and the second portion of the manifest file to a client device.

27. The device of claim 26, wherein the common characteristics for the set of representations include information indicative of at least one of a language for at least one of text and audio in the set of representations, a camera perspective of a scene for the set of representations, and a rating for the set of representations, wherein the rating describes content suitability for a particular audience.

28. The device of claim 26, wherein the one or more processors are configured to receive, from the client device, a request for the second portion of the manifest file, send the second portion of the manifest file to the client device in response to the request, receive, from the client device, a request for at least a portion of the second set of multimedia data from the location indicated by the second portion of the manifest file, and send, in response to the request for the portion of the second set of multimedia data, the requested portion of the second set of multimedia data to the client device.

29. The device of claim 28, wherein the multimedia content corresponding to the first portion of the manifest file comprises requested multimedia content, and wherein the one or more processors are configured to receive, from the client device, user information including at least one of a user identifier, user preferences for advertisements, and user demographic information, and select targeted advertisement multimedia data based on the received user information, wherein the selected targeted advertisement multimedia data is associated with the second portion of the manifest file.

30. The device of claim 28, wherein the second portion of the manifest file comprises a different portion of the manifest file than the first portion, and wherein the first set of multimedia data and the second set of multimedia data comprise the same set of multimedia data.

31. The device of claim 26, wherein the one or more processors are configured to send, before sending the at least first and second portions of the manifest file, information indicative of a profile to which the manifest file corresponds to the client device, and receive a request for the manifest file from the client device after sending the information indicative of the profile.

32. The device of claim 26, wherein the device comprises at least one of:

an integrated circuit;

a microprocessor; and

a wireless communication device that includes the one or more processors.

33. A computer program product comprising a non-transitory computer-readable storage medium having stored thereon instructions that, when executed, cause a processor of a device for providing multimedia data to:

obtain a set of representations of multimedia content having one or more common characteristics, wherein each of the representations in the set have individual coding characteristics separate from the common characteristics;

obtain a manifest file for the multimedia content, wherein the manifest file includes information indicative of the representations in the set, information indicative of the common characteristics for the set of representations, information indicative of the coding characteristics for each of the representations in the set, and information indicative of locations of a first set of multimedia data of at least one of the representations, and wherein the manifest file comprises a first portion and a second, different portion, wherein the first portion of the manifest file includes the information indicative of the representations in the set, the information indicative of the common characteristics, and information indicative of a location of the second portion of the manifest file, wherein the second portion includes information indicative of a location of a second set of multimedia data, wherein the manifest file is separate from the representations, and wherein the representations included in each of the sets of representations comprise switchable representations that are alternatives to each other and that each represent the same multimedia content; and

send at least the first portion and the second portion of the manifest file to a client device.

34. The computer program product of claim 33, wherein the common characteristics for the set of representations include information indicative of at least one of a language for at least one of text and audio in the set of representations, a camera perspective of a scene for the set of representations, and a rating for the set of representations, wherein the rating describes content suitability for a particular audience.

35. The computer program product of claim 33, further comprising instructions that cause the processor to:

receive, from the client device, a request for the second portion of the manifest file;

send the second portion of the manifest file to the client device in response to the request;

receive, from the client device, a request for at least a portion of the second set of multimedia data from the location indicated by the second portion of the manifest file; and

send, in response to the request for the portion of the second set of multimedia data, the requested portion of the second set of multimedia data to the client device.

36. The computer program product of claim 35, wherein the multimedia content corresponding to the first portion of the manifest file comprises requested multimedia content, further comprising instructions that cause the processor to:

receive, from the client device, user information including at least one of a user identifier, user preferences for advertisements, and user demographic information; and

select targeted advertisement multimedia data based on the received user information, wherein the selected targeted advertisement multimedia data is associated with the second portion of the manifest file.

37. The computer program product of claim 35, wherein the second portion of the manifest file comprises a different portion of the manifest file than the first portion, and wherein the first set of multimedia data and the second set of multimedia data comprise the same set of multimedia data.