Systems and methods providing metadata for media streaming

• US 10,135,896 B1
• Filed: 02/24/2014
• Issued: 11/20/2018
• Est. Priority Date: 02/24/2014
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Tiwary, Shinjan Kumar, Chakrovorthy, Sreeram Raju, Rai, Sapna Jayaram
• Primary Examiner(s): Lazaro, David R
• Assistant Examiner(s): Shitayewoldetadik, Berhanu
• Application Number:
  US14/187,768
• Time in Patent Office: 1,730 Days
• Field of Search: 709219, 709231, 709214, 709232, 709245, 375240, 725219
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/752 adapting media to network capabilities
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04L 65/80 Responding to QoS

Abstract

Media devices streaming video from a server may use techniques to maintain presentation of content to one or more users. Described herein are systems and methods for determining which metadata to provide to the media devices for maintaining presentation of the content. The determination of which metadata to provide may be based on various factors such as the capabilities of the media device, the expected quality of service, and the amount of bandwidth between the server and the media device.

Claims

1. A system for providing video content comprising:

a communication interface;

a memory storing computer-executable instructions; and

a processor communicatively coupled to the communication interface and the memory, the processor configured to execute the computer-executable instructions to;

access capability data indicating one or more capabilities of a media device;

access bandwidth data indicating an amount of bandwidth between a server and the media device;

access quality of service data indicating a requested quality of service associated with the video content streamed to the media device;

access a metadata datastore storing a plurality of metadata associated with the video content, wherein the plurality of metadata includes first metadata and second metadata;

generate resource consumption data indicating an amount of resources consumed to perform computational tasks;

in response to the amount of resources consumed as indicated by the resource consumption data being equal to or less than a threshold amount of resources, generate metadata comprising information associated with video frames of the video content including transformation data by selecting at least one of the first metadata or the second metadata, wherein the selection is based on the one or more capabilities of the media device, the amount of bandwidth and the requested quality of service; and

provide the metadata to the media device to maintain presentation of the video content by the media device using the metadata including the transformation data that includes information that indicates a transform to apply to a video frame for generation of one or more interpolated frames for adapting transmission of the video content to conditions affecting the presentation of the video content.

2. The system of claim 1, wherein:

the first metadata is selected in response to;

the amount of bandwidth being equal to or greater than a first threshold amount; and

the requested quality of service comprising a first requested quality of service;

the second metadata is selected in response to;

the amount of bandwidth being equal to or greater than a second threshold amount; and

the requested quality of service comprising a second requested quality of service; and

the first requested quality of service is different from the second requested quality of service.

3. The system of claim 1, wherein the processor is further configured to execute the computer-executable instructions to:

provide, at a first quality bitrate stream, the video content to the media device; and

generate transition data representative of a determination for the server to switch from providing the video content at the first quality bitrate stream to a second quality bitrate stream, wherein;

the metadata includes the transition data; and

the media device is configured to use the transition data to generate the one or more interpolated frames.

4. A method comprising:

accessing bandwidth data indicating an amount of bandwidth between a server and a media device;

accessing quality of service data indicating a requested quality of service associated with video content provided to the media device;

accessing a metadata datastore storing a plurality of metadata associated with the video content;

selecting metadata associated with video frames of the video content including transformation data, the metadata based on at least one of the requested quality of service, or the bandwidth data;

storing the metadata in the metadata datastore;

accessing the metadata datastore storing the metadata associated with the video frames of the video content; and

providing the metadata to the media device to maintain presentation of the video content by the media device using the metadata including the transformation data that includes information that indicates a transform to apply to a video frame for generation of one or more interpolated frames for adapting transmission of the video content to conditions affecting the presentation of the video content provided to the media device.

5. The method of claim 4, wherein an amount of the plurality of metadata is selected in response to a determination that the amount of bandwidth is greater than or equal to a threshold amount.

6. The method of claim 4, wherein:

the metadata comprises first metadata and second metadata, the first metadata is associated with a first priority and the second metadata is associated with a second priority;

the first metadata is selected in response to a determination that the first priority is higher than the second priority; and

the second metadata is selected in response to a determination that the second priority is higher than the first priority.

7. The method of claim 4, wherein:

the metadata comprises first metadata and second metadata, the first metadata is associated with a first priority and the second metadata is associated with a second priority;

at a first time, the first priority is higher than the second priority;

at a second time, the second priority is higher than the first priority in response to a determination that the amount of bandwidth is less than or equal to a threshold amount; and

the second metadata is selected in response to a determination that the second priority is higher than the first priority.

8. The method of claim 4, further comprising accessing capability data indicative of one or more capabilities of the media device, the metadata comprises first metadata and second metadata, wherein the first metadata is selected in response to a determination that the one or more capabilities include at least one or more predetermined capabilities.

9. The method of claim 4, wherein:

the requested quality of service comprises one of a first quality bitrate stream and a second quality bitrate stream;

the metadata comprises first metadata and second metadata, the first metadata and the second metadata are selected in response to a determination that the requested quality of service comprises the first quality bitrate stream; and

the first metadata is selected in response to a determination that the requested quality of service comprises the second quality bitrate stream.

10. The method of claim 4, further comprising:

generating resource consumption data including a value indicating one or more amounts of one or more resources consumed to perform computational tasks; and

in response to a determination that an amount of resources consumed as indicated by the resource consumption data is greater than a threshold amount of resources consumed;

terminate providing the video content to the media device; and

provide the metadata to the media device.

11. The method of claim 4, further comprising providing the video content to the media device, wherein:

the metadata comprises first metadata and second metadata, the first metadata comprises motion vector data comprising information about motion of one or more objects depicted in the video content; and

the media device uses the motion vector data to generate interpolated frames.

12. The method of claim 4, wherein the metadata comprises first metadata and second metadata, the first metadata comprises the transformation data indicative of the transform to apply to the video frame to generate the one or more interpolated frames, wherein the transform comprises one of a Discrete Cosine Transform (“

DCT”

), a wavelet transform, and a Karhunen-Loè

ve Transform (“

KLT”

).

13. The method of claim 12, wherein the transform is determined based on the requested quality of service associated with the video content received by the media device.

14. The method of claim 4, wherein the media device is configured to use the metadata to generate interpolated frames in response to a determination that a dependent frame is unavailable for presentation.

15. The method of claim 4, further comprising providing the video content to the media device, wherein the video content is compatible with at least a portion of the MPEG-2 standard promulgated by the Motion Pictures Experts Group (“

MPEG”

) or the H.264 standard developed by the Video Coding Experts Group and the MPEG, wherein the metadata is provided to the media device separate from the video content.

16. The method of claim 4, wherein:

the metadata comprises first metadata and second metadata, the first metadata is selected in response to;

the amount of bandwidth being equal to or greater than a first bandwidth threshold amount; and

the requested quality of service comprising a first requested quality of service;

the second metadata is selected in response to;

the amount of bandwidth being equal to or greater than a second bandwidth threshold amount; and

the requested quality of service comprising a second requested quality of service; and

the first requested quality of service is different from the second requested quality of service.

17. A media device comprising:

a communication interface;

a memory storing computer-executable instructions; and

a processor communicatively coupled to the communication interface and the memory, the processor configured to execute the computer-executable instructions to;

provide capability data to a server, wherein the capability data indicates one or more capabilities of the media device;

provide quality of service data to the server, wherein the quality of service data indicates a requested quality of service associated with video content provided to the media device;

receive metadata comprising information associated with video frames of the video content including transformation data, the metadata based on at least one of the capability data, the bandwidth data, and the requested quality of service;

use the metadata including the transformation data that includes information that indicates a transform to apply to a video frame to generate interpolated frames for adapting to conditions affecting presentation of the video content to maintain the presentation of the video content provided to the media device; and

send the interpolated frames for presentation at a display device.

18. The media device of claim 17, wherein:

the metadata comprises first metadata and second metadata, the first metadata comprises scene change data indicating a frame in which a video scene change occurs; and

the processor is further configured to execute the computer-executable instructions to use the scene change data to determine whether to generate the interpolated frames.

19. The media device of claim 17, wherein:

the metadata comprises quantization data representative of quantization factors corresponding to coefficients of the transform used to encode the video content; and

the processor is further configured to execute the computer-executable instructions to use the quantization data to generate the interpolated frames.

20. The media device of claim 17, wherein:

the metadata comprises first metadata and second metadata, the first metadata comprises motion vector data comprising information about motion of one or more objects depicted in the video content; and

the media device uses the motion vector data to generate the interpolated frames.