Link-aware streaming adaptation

• US 10,721,715 B2
• Filed: 02/27/2014
• Issued: 07/21/2020
• Est. Priority Date: 03/01/2013
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Ramamurthi, Vishwanath, Oyman, Ozgur
• Primary Examiner(s): Louie, Oscar A
• Assistant Examiner(s): Gidado, Oluwatosin M
• Application Number:
  US14/763,123
• Publication Number:
  US 20150373077A1
• Time in Patent Office: 2,336 Days
• Field of Search: 709219
• CPC Class Codes: H04B 7/0452 Multi-user MIMO systems
• CPC Class Codes: H04B 7/0486 taking channel rank into account
• CPC Class Codes: H04B 7/0626 Channel coefficients, e.g. channel state information [CSI]
• CPC Class Codes: H04J 3/12 Arrangements providing for calling or supervisory signals
• CPC Class Codes: H04L 12/184 with heterogeneous receivers, e.g. layered multicast
• CPC Class Codes: H04L 25/03 Shaping networks in transmitter or receiver, e.g. adaptive shaping networks
• CPC Class Codes: H04L 45/74 Address processing for routing
• CPC Class Codes: H04L 5/0037 Inter-user or inter-terminal allocation
• CPC Class Codes: H04L 65/4038 with floor control
• CPC Class Codes: H04L 65/65 Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04M 15/8044 Least cost routing
• CPC Class Codes: H04W 12/02 Protecting privacy or anonymity, e.g. protecting personally identifiable information [PII]
• CPC Class Codes: H04W 12/04 Key management, e.g. using generic bootstrapping architecture [GBA]
• CPC Class Codes: H04W 16/14 Spectrum sharing arrangements between different networks
• CPC Class Codes: H04W 24/02 Arrangements for optimising operational condition
• CPC Class Codes: H04W 24/08 Testing, supervising or monitoring using real traffic
• CPC Class Codes: H04W 24/10 Scheduling measurement reports ; Arrangements for measurement reports
• CPC Class Codes: H04W 36/1446 wherein at least one of the networks is unlicensed
• CPC Class Codes: H04W 36/165 : for reducing network power consumption H04W36/18 - H04W36/22 take precedence
• CPC Class Codes: H04W 36/22 : for handling the traffic
• CPC Class Codes: H04W 4/06 : Selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS]; Services to user groups; One-way selective calling services
• CPC Class Codes: H04W 4/10 : Push-to-Talk [PTT] or Push-On-Call services
• CPC Class Codes: H04W 4/80 : Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication
• CPC Class Codes: H04W 48/10 : using broadcasted information
• CPC Class Codes: H04W 48/20 : Selecting an access point
• CPC Class Codes: H04W 52/244 : Interferences in heterogeneous networks, e.g. among macro and femto or pico cells or other sector / system interference [OSI]
• CPC Class Codes: H04W 72/044 : based on the type of the allocated resource
• CPC Class Codes: H04W 72/20 : Control channels or signalling for resource management
• CPC Class Codes: H04W 72/30 : Resource management for broadcast services
• CPC Class Codes: H04W 72/542 : using measured or perceived quality
• CPC Class Codes: H04W 72/543 : based on requested quality, e.g. QoS
• CPC Class Codes: H04W 72/56 : based on priority criteria
• CPC Class Codes: H04W 76/10 : Connection setup
• CPC Class Codes: H04W 76/14 : Direct-mode setup
• CPC Class Codes: H04W 76/20 : Manipulation of established connections
• CPC Class Codes: H04W 76/23 : Manipulation of direct-mode connections
• CPC Class Codes: H04W 76/27 : Transitions between radio resource control [RRC] states
• CPC Class Codes: H04W 76/30 : Connection release
• CPC Class Codes: H04W 84/18 : Self-organising networks, e.g. ad-hoc networks or sensor networks
• CPC Class Codes: H04W 88/02 : Terminal devices
• CPC Class Codes: H04W 88/06 : adapted for operation in multiple networks or having at least two operational modes, e.g. multi-mode terminals
• CPC Class Codes: H04W 88/08 : Access point devices
• CPC Class Codes: Y02B 70/30 : Systems integrating technologies related to power network operation and communication or information technologies for improving the carbon footprint of the management of residential or tertiary loads, i.e. smart grids as climate change mitigation technology in the buildings sector, including also the last stages of power distribution and the control, monitoring or operating management systems at local level
• CPC Class Codes: Y02D 30/70 : in wireless communication networks

Abstract

Technology to provide link aware streaming adaptation is disclosed. In an example, a mobile device can include computer circuitry configured to: receive a manifest file for an HTTP adaptive stream from a node; determine a physical layer goodput of the mobile device with the node; and select a representation in the manifest file for a selected period based, at least in part, on the physical layer goodput.

Claims

1. A mobile device operable to receive hyper-text transfer protocol (HTTP) adaptive streaming (HAS), having computer circuitry configured to:

receive a manifest file for an HTTP adaptive stream from a node;

determine a physical layer goodput, R_i^PHY, of the mobile device with the node, wherein the physical layer goodput is given by;

R_i^PHY=X_i/τ

, wherein X_i is the number of successfully received bits in the i^th video frameslot and τ

is the video frame duration;

determine one or more frame levels for the mobile device;

determine a client state (S_i) for the mobile device based on the one or more frame levels; and

select a number of frames (N_i) and a representation level (Q_i) for the HAS in the manifest file for a selected period based, at least in part, on the one or more frame levels for the mobile device, the client state (S_i) for the mobile device, and the physical layer goodput,

wherein N_i and Q_i are selected to bound a delay of the HAS for a selected quality level at startup based in part on a first threshold value corresponding to a threshold number of frames to start playback of the HAS and a second threshold value corresponding to a target time to start playback, wherein the first threshold value and the second threshold value are selected to obtain a tradeoff between video quality and a start-up delay.

2. The computer circuitry of claim 1, wherein the computer circuitry is further configured to:

request a selected number of segments from the node that are in the representation, based, at least in part, on the physical layer goodput.

3. The computer circuitry of claim 1, wherein the computer circuitry is further configured to:

select the representation for the selected period based, at least in part, on one of the physical layer goodput or a throughput estimate from a transport layer or a throughput estimate from an application layer.

4. The computer circuitry of claim 1, wherein the computer circuitry is further configured to:

determine an average physical layer goodput over a selected number of previously received frame slots; and

request a selected number of segments to be downloaded from the node within a predetermined period based on the average physical layer goodput.

5. The computer circuitry of claim 1, wherein the computer circuitry is further configured to:

receive quality of service (QoS) parameters from the node, including a maximum bit rate (MBR) and a guaranteed bit rate (GBR); and

request a selected number of segments from the node based, at least in part, on the physical layer goodput, the MBR, and the GBR.

6. The computer circuitry of claim 1, wherein the computer circuitry is further configured to select, at a startup of the HTTP adaptive stream:

a predetermined number of segments to fill a buffer prior to operating the stream at the mobile device, based, at least in part, on the physical layer goodput;

a representation for the selected period and a predetermined number of segments to provide a desired quality of the HTTP adaptive stream at the mobile device, based, at least in part, on an average physical layer goodput;

a representation for the selected period and a predetermined number of segments to provide a minimized startup delay at the mobile device based, at least in part, on the average physical layer goodput;

or

a representation for the selected period and a predetermined number of segments to provide a delay bounded quality optimization at the mobile device based, at least in part, on the average physical layer goodput.

7. The computer circuitry of claim 1, wherein the computer circuitry is further configured to select the representation and request a selected number of segments during a steady state playback of the HTTP adaptive stream by:

determining a ratio of an average of the physical layer goodput to the selected representation; and

requesting more than one segment to be delivered per segment duration to build up a number of segments in a buffer of the mobile device when additional link bandwidth is available during the steady state playback, based, at least in part, on the average physical layer goodput.

8. The computer circuitry of claim 7, wherein the computer circuitry is further configured to:

determine that the average physical layer goodput is increasing before requesting the more than one segment to be delivered per segment duration; and

select a limit on a number of segments to be delivered per segment duration during the steady state playback.

9. The computer circuitry of claim 1 wherein the manifest file for the HTTP adaptive stream is:

a media presentation description for a dynamic adaptive streaming over HTTP (DASH) adaptation set;

or

metadata embedded in a 3GP file format.

10. A method for receiving hyper-text transfer protocol (HTTP) adaptive streaming at a mobile device, the method comprising:

receiving a manifest file for an HTTP adaptive stream (HAS) from a node;

determining a physical layer goodput, R_i^PHY, of the mobile device with the node, wherein the physical layer goodput is given by;

R_i^PHY=X_i/τ

, wherein X_i is the number of successfully received bits in the i^th video frameslot and τ

is the video frame duration;

determining one or more frame levels for the mobile device;

determining a client state (S_i) for the mobile device based on the one or more frame levels; and

selecting a number of frames (N_i) and a representation level (Q_i) for the HAS based, at least in part, on the one or more frame levels for the mobile device, the client state (S_i) for the mobile device, and the physical layer goodput,

wherein N_i and Q_i are selected to bound a delay of the HAS for a selected quality level at startup based in part on a first threshold value corresponding to a threshold number of frames to start playback of the HAS and a second threshold value corresponding to a target time to start playback, wherein the first threshold value and the second threshold value are selected to obtain a tradeoff between video quality and a start-up delay.

11. The method of claim 10, wherein determining the bitrate further comprises:

selecting a representation in the manifest file for a selected period based, at least in part, on the physical layer goodput;

or

requesting a selected number of segments from the node that are in the representation, based, at least in part, on the physical layer goodput.

12. The method of claim 11, wherein requesting the number of segments from the node further comprises:

sending an HTTP get request from the mobile device to a web server, wherein a frequency of HTTP get requests is selected based, at least in part, on the physical layer goodput.

13. The method of claim 10, further comprising:

determining an average physical layer goodput over a selected number of previously received frame slots; and

requesting an additional number of segments to be delivered to the mobile device per segment duration to build up a number of segments in a buffer of the mobile device when the average physical layer goodput is increasing.

14. The method of claim 10, further comprising using transport layer or application layer throughput information in place of the physical layer goodput to determine the bitrate to request for the HAS or the desired quality level to provide for the HAS based on:

a link condition between the mobile device and the node;

a network congestion level of the node;

a client state;

or

a rate at which a buffer of the mobile device is being filled with the HAS.

15. The method of claim 10, further comprising receiving the physical layer goodput information periodically at a client operating on the mobile device from an application programming interface (API) with a radio component of the mobile device.

16. The method of claim 10, further comprising:

receiving quality of service (QoS) parameters from the node, including a maximum bit rate (MBR) and a guaranteed bit rate (GBR); and

requesting a selected number of segments from the node based, at least in part, on the physical layer goodput, the MBR, and the GBR.

17. At least one non-transitory machine readable storage medium comprising a plurality of instructions adapted to be executed to implement the method of claim 10.

18. A user equipment (UE) operable to perform link aware streaming adaptation, having computer circuitry configured to:

determine a physical layer goodput, R_i^PHY, and a higher layer throughput estimate for a data link of the UE with a web server, wherein the physical layer goodput is given by;

R_i^PHY=X_i/τ

, wherein X_i is the number of successfully received bits in the i^th video frameslot and τ

is the video frame duration;

determine one or more frame levels for the UE;

determine a client state (S_i) for the UE based on the one or more frame levels;

receive a manifest file for an HTTP adaptive stream (HAS) from the web server;

determine whether to use the physical layer goodput or the higher layer throughput estimate based on radio link conditions of the data link; and

select a number of frames (N_i) and a representation level (Q_i) for the HAS based on the one or more frame levels for the UE, the client state (S_i) for the UE, and the selected physical layer goodput,

wherein N_i and Q_i are selected to bound a delay of the HAS for a selected quality level at startup based in part on a first threshold value corresponding to a threshold number of frames to start playback of the HAS and a second threshold value corresponding to a target time to start playback, wherein the first threshold value and the second threshold value are selected to obtain a tradeoff between video quality and a start-up delay.

19. The user equipment of claim 18, further configured to perform a quality-optimized startup by selecting a next available video adaptation rate when enough bandwidth is available based on the physical layer goodput.

20. The user equipment of claim 18, further configured to select the number of frames N_i, and the representation level, Q_i, to optimize a quality of the HAS at startup, using:

21. The user equipment of claim 18, further configured to select the number of frames N_i, and the representation level Q_i, to bound a delay of the HAS for a selected quality level at startup, using:

22. The user equipment of claim 21, wherein the threshold values B_thresh^StartUp and T_thresh^StartUP are selected to obtain the tradeoff between video quality and the start-up delay.