Energy-aware multimedia adaptation for streaming and conversational services

• US 9,629,131 B2
• Filed: 09/30/2013
• Issued: 04/18/2017
• Est. Priority Date: 09/28/2012
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Intel Corporation
• Inventors: Ozgur, Oyman
• Primary Examiner(s): Nguyen, Dustin
• Application Number:
  US14/041,476
• Publication Number:
  US 20140095670A1
• Time in Patent Office: 1,296 Days
• Field of Search: 709231, 709217, 348583, 725 62, 37524002, 704233
• CPC Class Codes: H04B 17/318 Received signal strength
• CPC Class Codes: H04J 11/00 Orthogonal multiplex systems, e.g. using WALSH codesH04J13/00 takes precedence
• CPC Class Codes: H04J 11/0086 Search parameters, e.g. search strategy, accumulation length, range of search, thresholds code acquisition in DSSS H04B1/7075
• CPC Class Codes: H04J 3/1694 Allocation of channels in TDM/TDMA networks, e.g. distributed multiplexers Passive Optical Networks H04Q11/0062
• CPC Class Codes: H04L 1/1812 Hybrid protocols; Hybrid automatic repeat request [HARQ]
• CPC Class Codes: H04L 1/1861 Physical mapping arrangements for ACK signaling see also H04L5/0053
• CPC Class Codes: H04L 1/1864 ARQ related signaling H04L1/1607 takes precedence
• CPC Class Codes: H04L 1/1893 Physical mapping arrangements physical resource mapping in general H04L5/00
• CPC Class Codes: H04L 41/069 using logs of notifications; Post-processing of notifications
• CPC Class Codes: H04L 41/5032 Generating service level reports
• CPC Class Codes: H04L 43/16 Threshold monitoring
• CPC Class Codes: H04L 5/001 the frequencies being arranged in component carriers
• CPC Class Codes: H04L 5/0035 Resource allocation in a cooperative multipoint environment
• CPC Class Codes: H04L 5/0055 Physical resource allocation for ACK/NACK for physical mapping arrangements in ARQ protocols H04L1/1861
• CPC Class Codes: H04L 5/0057 Physical resource allocation for CQI
• CPC Class Codes: H04L 5/0073 Allocation arrangements that take into account other cell interferences for intercell interference mitigation or co-ordination in orthogonal multiplex systems H04J11/005
• CPC Class Codes: H04L 5/0091 Signaling for the administration of the divided path
• CPC Class Codes: H04L 5/14 Two-way operation using the same type of signal, i.e. duplex
• CPC Class Codes: H04L 65/60 Network streaming of media packets
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/65 : Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
• CPC Class Codes: H04L 65/762 : at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04L 67/10 : in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04W 24/02 : Arrangements for optimising operational condition
• CPC Class Codes: H04W 24/08 : Testing, supervising or monitoring using real traffic
• CPC Class Codes: H04W 24/10 : Scheduling measurement reports ; Arrangements for measurement reports
• CPC Class Codes: H04W 28/0205 : at the air interface dynamic wireless traffic scheduling H04W72/12
• CPC Class Codes: H04W 28/0215 : based on user or device properties, e.g. MTC-capable devices services for machine-to-machine communication [M2M] or machine type communication [MTC] H04W4/70; wireless resource selection or allocation plan definition based on terminal or device properties H04W72/51
• CPC Class Codes: H04W 28/0221 : power availability or consumption
• CPC Class Codes: H04W 28/0252 : per individual bearer or channel dynamic wireless traffic scheduling H04W72/12
• CPC Class Codes: H04W 28/0268 : using specific QoS parameters for wireless networks, e.g. QoS class identifier [QCI] or guaranteed bit rate [GBR] negotiating SLA or negotiating QoS H04W28/24
• CPC Class Codes: H04W 28/16 : Central resource management; Negotiation of resources or communication parameters, e.g. negotiating bandwidth or QoS [Quality of Service]
• CPC Class Codes: H04W 36/0061 : of neighbour cell information
• CPC Class Codes: H04W 36/0066 : of control information between different types of networks in order to establish a new radio link in the target network
• CPC Class Codes: H04W 36/0083 : Determination of parameters used for hand-off, e.g. generation or modification of neighbour cell lists
• CPC Class Codes: H04W 36/0085 : Hand-off measurements
• CPC Class Codes: H04W 36/0088 : Scheduling hand-off measurements
• CPC Class Codes: H04W 36/22 : for handling the traffic
• CPC Class Codes: H04W 4/70 : Services for machine-to-machine communication [M2M] or machine type communication [MTC]
• CPC Class Codes: H04W 40/005 : Routing actions in the presence of nodes in sleep or doze mode
• CPC Class Codes: H04W 40/246 : Connectivity information discovery
• CPC Class Codes: H04W 48/14 : using user query or user detection
• CPC Class Codes: H04W 48/16 : Discovering, processing access restriction or access information
• CPC Class Codes: H04W 48/18 : Selecting a network or a communication service
• CPC Class Codes: H04W 48/20 : Selecting an access point
• CPC Class Codes: H04W 52/0209 : in terminal devices terminal devices per se H04W88/02
• CPC Class Codes: H04W 52/0212 : managed by the network, e.g. network or access point is master and terminal is slave
• CPC Class Codes: H04W 52/0225 : using monitoring of external events, e.g. the presence of a signal
• CPC Class Codes: H04W 52/0235 : where the received signal is a power saving command
• CPC Class Codes: H04W 52/0258 : controlling an operation mode according to history or models of usage information, e.g. activity schedule or time of day
• CPC Class Codes: H04W 52/0261 : managing power supply demand, e.g. depending on battery level
• CPC Class Codes: H04W 52/04 : TPC
• CPC Class Codes: H04W 52/14 : Separate analysis of uplink or downlink
• CPC Class Codes: H04W 72/02 : Selection of wireless resources by user or terminal
• CPC Class Codes: H04W 72/044 : based on the type of the allocated resource
• CPC Class Codes: H04W 72/0453 : Resources in frequency domain, e.g. a carrier in FDMA
• CPC Class Codes: H04W 72/20 : Control channels or signalling for resource management
• CPC Class Codes: H04W 72/21 : in the uplink direction of a wireless link, i.e. towards the network
• CPC Class Codes: H04W 72/23 : in the downlink direction of a wireless link, i.e. towards a terminal
• CPC Class Codes: H04W 72/30 : Resource management for broadcast services
• CPC Class Codes: H04W 72/52 : based on load
• CPC Class Codes: H04W 72/541 : using the level of interference
• CPC Class Codes: H04W 74/002 : Transmission of channel access control information
• CPC Class Codes: H04W 74/004 : in the uplink, i.e. towards network
• CPC Class Codes: H04W 76/12 : Setup of transport tunnels
• CPC Class Codes: H04W 76/15 : Setup of multiple wireless link connections
• CPC Class Codes: H04W 76/16 : Involving different core network technologies, e.g. a packet-switched [PS] bearer in combination with a circuit-switched [CS] bearer
• CPC Class Codes: H04W 76/23 : Manipulation of direct-mode connections
• CPC Class Codes: H04W 76/27 : Transitions between radio resource control [RRC] states
• CPC Class Codes: H04W 76/28 : Discontinuous transmission [DTX]; Discontinuous reception [DRX]
• CPC Class Codes: H04W 76/30 : Connection release
• CPC Class Codes: H04W 8/08 : Mobility data transfer
• CPC Class Codes: H04W 80/10 : adapted for application session management, e.g. SIP [Session Initiation Protocol] connection management H04W76/00; arrangements for session management H04L67/14
• CPC Class Codes: H04W 84/042 : Public Land Mobile systems, e.g. cellular systems
• CPC Class Codes: H04W 84/12 : WLAN [Wireless Local Area Networks]
• CPC Class Codes: H04W 88/02 : Terminal devices
• CPC Class Codes: H04W 88/06 : adapted for operation in multiple networks or having at least two operational modes, e.g. multi-mode terminals
• CPC Class Codes: H04W 88/08 : Access point devices
• CPC Class Codes: H04W 88/12 : Access point controller devices
• CPC Class Codes: H04W 88/14 : Backbone network devices
• CPC Class Codes: H04W 88/16 : Gateway arrangements
• CPC Class Codes: H04W 88/18 : Service support devices; Network management devices
• CPC Class Codes: Y02D 30/70 : in wireless communication networks

Abstract

Technology to provide is disclosed. In an example, a user equipment (UE) can include computer circuitry configured to: receive a manifest file for an HTTP adaptive stream from a node; identify an energy characterization in the manifest file for a plurality of representations for a selected period; select a representation for the selected period having a desired energy characterization; and request segments from the node that are in the representation having the desired energy characterization.

Claims

1. A mobile device operable to provide hyper-text transfer protocol (HTTP) adaptive streaming, having computer circuitry hardware configured to:

receive a manifest file for an HTTP adaptive stream from a node;

identify an energy characterization in the manifest file for each of a plurality of representations for a selected period, wherein the energy characterization is a determination of an amount of energy or power used by the mobile device to decode and display the HTTP adaptive stream of a representation of the plurality of representations for the selected period and includes at least a first attribute defining a minimum energy value for each of the plurality of representations for the selected period and a second attribute defining a maximum energy level for each of the plurality of representations for the selected period;

select a representation for the selected period having a desired energy characterization; and

request segments from the node that are in the representation having the desired energy characterization.

2. The computer circuitry hardware of claim 1 wherein the computer circuitry is further configured to:

determine a power budget of the mobile device to decode and perform the HTTP adaptive stream; and

select the representation for the selected period having the desired energy characterization for the power budget.

3. The computer circuitry of claim 1 wherein the energy characterization for the HTTP adaptive stream is included in:

a media presentation description for a dynamic adaptive streaming over HTTP (DASH) adaptation set;

or

metadata embedded in a 3GP file format.

4. The computer circuitry hardware of claim 1, wherein the computer circuitry is further configured to communicate, to a content server, via the node, an energy characterization for power consumed during reception, decoding, or rendering of the HTTP adaptive stream.

5. The computer circuitry hardware of claim 4, wherein the mobile device receives a representation having a different energy characterization from the content server, based on the energy characterization reported to the content server.

6. The computer circuitry hardware of claim 1, wherein the computer circuitry is further configured to:

identify an energy characterization in the manifest file for a plurality of sub-representations for the selected period; and

request sub-segments from the node that are in at least one of the sub-representations and have a desired energy characterization.

7. The computer circuitry hardware of claim 1, wherein the computer circuitry is further configured to switch to a different representation having a different energy characterization at a selected access point in the HTTP adaptive stream to change an amount of power consumed by the mobile device to decode and perform the HTTP adaptive stream.

8. The computer circuitry hardware of claim 1, wherein the media device is in communication with a metadata management server configured to communicate the energy characterization information.

9. A media server operable to provide hyper-text transfer protocol (HTTP) adaptive streaming, having computer circuitry hardware configured to:

communicate energy characterization metadata to a mobile device over a byte range for a plurality of representations of an HTTP adaptive stream, wherein the energy characterization is a determination of an amount of energy or power used by the mobile device to decode and display the HTTP adaptive stream of a representation of the plurality of representations for the selected period and includes at least a first attribute defining a minimum energy value for each of the plurality of representations for the selected period and a second attribute defining a maximum energy level for each of the plurality of representations for the selected period;

receive a request for segments of the HTTP adaptive stream in the byte range in a representation selected from the plurality of representations, wherein the representation is selected based, at least in part, on the energy characterization metadata; and

communicate the requested segments in the byte range to the mobile device.

10. The computer circuitry hardware of claim 9, wherein the computer circuitry is further configured to determine the energy characterization metadata over the byte range based on content-specific information comprising frame pixel histograms, a maximum frame-pixel value, a block coded type, or deblocking filtering numbers.

11. The computer circuitry hardware of claim 9, wherein the computer circuitry is further configured to determine a minimum and a maximum power level for segments over the byte range.

12. The computer circuitry hardware of claim 9, wherein the byte range is for a period, an adaptation set, a representation, a segment or a sub-segment.

13. The computer circuitry hardware of claim 9, wherein the computer circuitry is further configured to provide a vectorized set of energy/power values to specify a minimum and a maximum energy level of each period, adaptation set, representation, segment or sub-segment in the energy characterization metadata.

14. The computer circuitry hardware of claim 9, wherein the manifest file for the HTTP adaptive stream is:

a media presentation description for a dynamic adaptive streaming over HTTP (DASH) adaptation set;

or

metadata embedded in a 3GP file format.

15. The computer circuitry hardware of claim 9, wherein the computer circuitry is further configured to receive the request for segments of the HTTP adaptive stream in the byte range, wherein the segments are located in an adaptation set that includes the plurality of representations, and at least one representation includes a plurality of segments.

16. The computer circuitry hardware of claim 9, wherein each representation contains a same media file over the byte range that is encoded differently than other representations, wherein the encoding includes a bit rate, a frame rate, a resolution, or a codec type.

17. The computer circuitry hardware of claim 9, wherein the computer circuitry is further configured to:

receive a quality of experience (QoE) report from the mobile device that includes a power metric associated with power that is consumed at the mobile device during reception, decoding, or rendering of the HTTP adaptive stream; and

communicate a different representation to the mobile device based on the power metric received in the QoE report.

18. The computer circuitry hardware of claim 9, wherein the computer circuitry is further configured to:

receive a request for segments of the HTTP adaptive stream in the byte range in a different representation selected from the plurality of representation, wherein the different representation is selected based, at least in part, on energy usage at the mobile device; and

communicate the requested segments in the byte range to the mobile device.