Power aware video decoding and streaming

• US 10,154,258 B2
• Filed: 07/09/2013
• Issued: 12/11/2018
• Est. Priority Date: 07/09/2012
• Status: Active Grant

• Current Assignee: VID SCALE Inc. (InterDigital, Inc.)
• Original Assignee: VID SCALE Inc. (InterDigital, Inc.)
• Inventors: He, Yuwen, Ye, Yan, He, Yong, McClellan, George W., Ryu, Eun
• Primary Examiner(s): Perez-Fuentes, Luis
• Application Number:
  US13/937,678
• Publication Number:
  US 20140010282A1
• Time in Patent Office: 1,981 Days
• Field of Search: 37524002
• CPC Class Codes: G06F 1/3212 Monitoring battery levels, e.g. power saving mode being initiated when battery voltage goes below a certain level
• CPC Class Codes: G06F 1/329 by task scheduling
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/613 for the control of the source by the destination control signals issued by the client directed to the server or network components specially adapted for selective content distribution H04N21/637
• CPC Class Codes: H04L 65/756 adapting media to device capabilities
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/127 Prioritisation of hardware or computational resources
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/156 Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
• CPC Class Codes: H04N 19/187 the unit being a scalable video layer
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 21/23439 for generating different versions
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/4424 Monitoring of the internal components or processes of the client device, e.g. CPU or memory load, processing speed, timer, counter or percentage of the hard disk space used error monitoring in general G06F11/30; arrangements for monitoring conditions of receiving stations in broadcast systems H04H60/32; diagnosis, testing or measuring for television receivers H04N17/04
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments
• CPC Class Codes: H04N 21/85406 involving a specific file format, e.g. MP4 format
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

Methods and systems are disclosed for a mobile device to decode video based on available power and/or energy. For example, the mobile device may receive a media description file (MDF) from for a video stream from a video server. The MDF may include complexity information associated with a plurality of video segments. The complexity information may be related to the amount of processing power to be utilized for decoding the segment at the mobile device. The mobile device may determine at least one power metric for the mobile device. The mobile device may determine a first complexity level to be requested for a first video segment based on the complexity information from the MDF and the power metric. The mobile device may dynamically alter the decoding process to save energy based on the detected power/energy level.

Claims

1. A method for a mobile device to request a video stream from a video server, the method comprising:

receiving a media description file (MDF) for the video stream, wherein the video stream is divided into a plurality of video segments, and wherein the MDF comprises video decoding complexity information associated with a first video segment, and wherein the video decoding complexity information for the first video segment indicates relative video decoding power consumption information for a plurality of complexity levels at which the first video segment can be requested;

calculating an amount of power consumed by the mobile device to decode a previous video segment at a reference complexity level of the plurality of complexity levels;

determining an amount of energy remaining in a battery of the mobile device;

calculating, using the video decoding complexity information, a power reduction ratio for the first video segment relative to the previous video segment at the reference complexity level;

determining to request the first video segment at a first complexity level of the plurality of complexity levels based on the amount of energy remaining in the battery of the mobile device, the power reduction ratio, and the amount of power consumed by the mobile device to decode the previous video segment at the reference complexity level; and

requesting the first video segment at the first complexity level from the video server.

2. The method as in claim 1, wherein the video decoding complexity information is provided per video segment in the MDF.

3. The method as in claim 1, wherein the MDF corresponds to a media presentation description (MPD) file.

4. The method as in claim 1, wherein the MDF further comprises bit rate information for the plurality of video segments, and the bit rate information indicates a plurality of bit rate levels at which at least the first video segment can be requested.

5. The method as in claim 1, wherein the relative video decoding power consumption information for the plurality of complexity levels at which the first video segment can be requested indicates relative amounts of energy that would be used to decode the first video segment at each of the plurality of complexity levels, and wherein the plurality of complexity levels correspond to a video encoder varying one or more encoding parameters in order to encode the first video segment, and the one or more encoding parameters correspond to at least one of precision of motion information, motion compensation block size, transform size, in-loop filters, or a cost threshold for skipping coefficient coding.

6. The method as in claim 1, further comprising estimating, based on the amount of energy remaining in the battery of the mobile device, a power available for decoding the first video segment, wherein determining to request the first video segment at the first complexity level of the plurality of complexity levels is further based on the estimated power available for decoding the first video segment.

7. The method as in claim 1, wherein determining to request the first video segment at the first complexity level comprises determining the first complexity level such that decoding the first video segment results in using approximately an amount of energy allocated for decoding the first video segment.

8. The method as in claim 1, wherein determining to request the first video segment at the first complexity level comprises selecting the first complexity level based on a power allocated for the previous video segment, a complexity level used to decode the previous video segment, and a power allocated for decoding the first video segment.

9. The method as in claim 1, further comprising determining whether a region of the first video segment comprises high frequency components that exceed a high frequency threshold, and applying a first interpolation filter during motion compensation for the region on condition that the high frequency components exceed the high frequency threshold or applying a second interpolation filter for motion compensation for the region on condition that the high frequency components do not exceed the high frequency threshold, wherein the second interpolation filter is associated with a lower cutoff frequency that the first interpolation filter.

10. The method as in claim 1, further comprising performing deblocking for reference pictures in the first video segment, and refraining from performing deblocking for non-reference pictures.

11. A mobile device for requesting a video stream from a video server based on power conditions, the mobile device comprising:

a transceiver configured to receive a media description file (MDF) for the video stream, wherein the video stream is divided into a plurality of video segments, and wherein the MDF comprises video decoding complexity information associated with a first video segment, and wherein the video decoding complexity information for the first video segment indicates relative video decoding power consumption information for a plurality of complexity levels at which the first video segment can be requested; and

a processor configured to;

calculate an amount of power consumed by the mobile device to decode a previous video segment at a reference complexity level of the plurality of complexity levels;

determine an amount of energy remaining in a battery of the mobile device;

calculate, using the video decoding complexity information, a power reduction ratio for the first video segment relative to the previous video segment at the reference complexity level; and

determine to request the first video segment at a first complexity level of the plurality of complexity levels based on the amount of energy remaining in the battery of the mobile device, the power reduction ratio, and the amount of power consumed by the mobile device to decode the previous video segment at the reference complexity level.

12. The mobile device as in claim 11, wherein the processor is further configured to determine a parameter or method to apply for decoding the first video segment based on the video decoding complexity information indicated in the MDF.

13. The mobile device as in claim 11, wherein the processor is further configured to determine an interpolation filter to apply for motion compensation of a given region based on a frequency analysis of a sampling of pixel within the given region.

14. The mobile device as in claim 13, wherein the processor is further configured to utilize a first interpolation filter with a lower cutoff frequency than a second interpolation filter for higher layer temporal prediction for regions with high frequency components below a specified threshold, and use the second interpolation filter for lower layer temporal prediction even if high frequency components of the lower layer are below the threshold.

15. The mobile device as in claim 13, wherein the processor is further configured to utilize a first interpolation filter with a lower cutoff frequency than a second interpolation filter for regions with high frequency components below a specified threshold except for at least inter blocks where a non-causal neighbor of the inter block comprise one or more intra coded blocks.

16. The mobile device as in claim 11, wherein the processor is further configured to determine a resolution to request for a subsequent segment based on the relative video decoding power consumption information indicated in the MDF, the amount of energy remaining in the battery of the mobile device, and the amount of power consumed by the mobile device to decode the previous video segment at the reference complexity level.

17. A method for a wireless transmit receive unit (WTRU) to request a video stream from a video server, the method comprising:

the WTRU requesting a media description file (MDF);

the WTRU receiving the MDF, the MDF comprising video decoding complexity information for decoding one or more video segments of the video stream, and the video decoding complexity information indicates relative video decoding power consumption information for a plurality of complexity levels at which video segments can be requested;

the WTRU calculating an amount of power consumed by the WTRU to decode a previous video segment at a reference complexity level of the plurality of complexity levels;

the WTRU determining an amount of energy remaining in a battery of the WTRU;

the WTRU calculating, using the video decoding complexity information, a power reduction ratio for a first video segment of the one or more video segments relative to the previous video segment at the reference complexity level;

the WTRU determining to request the first video segment of the one or more video segments at a first decoding complexity level based on the amount of energy remaining in the battery of the WTRU, the power reduction ratio, and the amount of power consumed by the WTRU to decode the previous video segment at the reference complexity level;

the WTRU requesting the first complexity level for the first video segment;

the WTRU re-evaluating the amount of energy remaining in the battery of the WTRU prior to requesting a second video segment of the video stream; and

the WTRU determining to request the second video segment at a second complexity level based on the re-evaluation of the amount of energy remaining in the battery of the WTRU.

18. The method as in claim 17, wherein the video decoding complexity information is provided on a per video segment basis.

19. The method as in claim 17, further comprising the WTRU estimating, based on the amount of energy remaining in the battery of the WTRU, a power available for decoding the first video segment, wherein determining to request the first video segment at the first complexity level of the plurality of complexity levels is further based on the estimated power available for decoding the first video segment.

20. The method as in claim 17, further comprising the WTRU reporting power usage statistics for various decoded segments to the video server, and the WTRU receiving a video segment at a given complexity based on the report.