Full-reference computation of mobile content quality of experience in real-time

• US 9,060,191 B2
• Filed: 06/20/2011
• Issued: 06/16/2015
• Est. Priority Date: 04/20/2011
• Status: Expired due to Fees

• Current Assignee: Empire Technology Development LLC (Allied Inventors Management, LLC)
• Original Assignee: Empire Technology Development LLC (Allied Inventors Management, LLC)
• Inventors: Chhaochharia, Nikhil, Ramasangu, Hariharan, Koomullil, George Paulose, Tulsian, Ashish
• Primary Examiner(s): Sheleheda, James R
• Application Number:
  US13/378,993
• Publication Number:
  US 20130133011A1
• Time in Patent Office: 1,457 Days
• Field of Search: 725/95, 725/96, 725/107
• US Class Current: 1/1
• CPC Class Codes: H04L 47/24 Traffic characterised by specific attributes, e.g. priority or QoS
• CPC Class Codes: H04L 47/25 with rate being modified by the source upon detecting a change of network conditions
• CPC Class Codes: H04N 21/2402 Monitoring of the downstream path of the transmission network, e.g. bandwidth available traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876
• CPC Class Codes: H04N 21/2408 Monitoring of the upstream path of the transmission network, e.g. client requests monitoring data switching networks utilization H04L43/0876; scheduling or organising the servicing of application requests H04L67/60
• CPC Class Codes: H04W 24/06 Testing, supervising or monitoring using simulated traffic
• CPC Class Codes: H04W 28/10 Flow control between communication endpoints
• CPC Class Codes: H04W 8/04 Registration at HLR or HSS [Home Subscriber Server]

Abstract

Technologies are generally described for Quality of Experience (QoE) computation in a wireless content transmission environment. Portions of audio and/or video content transmitted to user devices may be selected by a service provider/content provider/user device and transmitted back from the user device for real-time QoE computation. Transmission quality may be adjusted based on the real-time QoE computations. Alternatively, selected portions of the content may be stored for subsequent computation and improvement. Selected portions of the data may be frames marked as such and reference frames may be selected based on the marked frames.

Claims

1. A method to determine quality of experience (QoE) in a wireless system, the method comprising:

transmitting, by one of a service provider or a content provider, data to a user device, wherein the transmitted data is played back at the user device;

in response to a determination that sufficient bandwidth is available to receive one or more selected portions of the transmitted data from the user device through an upload channel,

receiving the one or more selected portions of the transmitted data that can be reliably received for real time QoE computation from the user device at one of the service provider or the content provider, wherein the one or more selected portions are selected on periodic intervals or based on a predefined algorithm on the user device;

in response to a determination that data upload and data download is shared through a same channel,

receiving the one or more selected portions of the transmitted data from the user device at a non-peak data transmission time period for the real time QoE computation at one of the service provider or the content provider such that additional bandwidth is released for upload purposes by downgrading a quality of transmissions;

performing, by one of the service provider or the content provider, the QoE computation using the one or more selected portions of the transmitted data;

performing, b one of the service provider or the content provider at least one of spatial and/or temporal alignment between an unimpaired reference frame of the one or more selected portions of the transmitted data and a corresponding frame of received portions of the transmitted data and

transmitting, by one of the service provider or the content provider, QoE computation results to a data store to subsequently analyze the QoE computation results and adjust QoE parameters based on the subsequent analysis of the QoE computation results.

2. The method according to claim 1, further comprising one or more of:

determining, by one of the service provider or the content provider, the one or more selected portions of the transmitted data;

transmitting, by one of the service provider or the content provider, information associated with the one or more selected portions of the transmitted data to the user device; and

using the one or more selected portions of the transmitted data to perform the QoE computation at one of the service provider or the content provider by comparing reference data portions of the transmitted data with the received portions of the transmitted data, wherein the transmitted data includes at least one of video data and/or audio data.

3. The method according to claim 2, wherein the reference data portions of the transmitted data are video frames, and wherein transmitting information associated with the reference data portions of the transmitted data includes marking the video frames to be selected by the user device.

4. The method according to claim 2, further comprising:

adjusting, by one of the service provider or the content provider, one or more quality of service (QoS) parameters based on the QoE computation in real time.

5. The method according to claim 2, further comprising:

adjusting, by one of the service provider or the content provider, one or more future quality of service (QoS) parameters based on the QoE computation.

6. The method according to claim 2, further comprising:

receiving at one of the service provider or the content provider the one or more selected portions of the transmitted data during one or more quiet periods; and

storing at one of the service provider or the content provider the received portions of the transmitted data for subsequent QoE computation, wherein the quiet periods include low bandwidth demand periods.

7. The method according to claim 2, wherein the QoE computation includes comparing each pixel of an unimpaired reference frame of the one or more selected portions of the transmitted data with each pixel of the corresponding frame of the received portions of the transmitted data.

8. The method according to claim 2, further comprising:

correlating results of the QoE computation with one or more of instantaneous QoS parameters, a location of the user device, a mobility of the user device, a signal strength at the user device, available memory at the user device, a processing capacity of the user device, and/or a load on communication network.

9. A method to enable computation of quality of experience (QoE) in a wireless system, the method comprising:

receiving data from one of a service provider and a content provider to be played back at a user device;

determining, by the user device, one or more selected portions of the received data to be transmitted to one of the service provider or the content provider for QoE computation;

capturing, by the user device, the one or more selected portions of the data to be transmitted to one of the service provider or the content provider;

in response to a determination that sufficient bandwidth is available to transmit the one or more selected portions of the received data including at least one of video data and/or audio data through an upload channel, transmitting,by the user device, a largest possible portion of the received data to the service provider that can be reliably transmitted for real time QoE computation at one of the service provider or the content provider, wherein the one or more selected portions are selected on periodic intervals or based on a predefined algorithm on the user device; and

in response to a determination that data upload and data download is shared through a same channel, transmitting, by the user device, the one or more selected portions of the received data at a non-peak data transmission time period for the real time QoE computation to one of the service provider or the content provider such that additional bandwidth is released for upload purposes by downgrading a quality of transmissions, wherein one of the service provider or the content provider is configured to;

perform at least one of spatial and/or temporal alignment between an unimpaired reference frame of the one or more selected portions of the received data received from one of the service provider and the content provider at the user device and a corresponding frame of the one or more selected portions of the received data transmitted from the user device to one of the service provider or the content provider; and

transmit QoE computation results to a data store to subsequently analyze the QoE computation results and adjust QoE parameters based on the subsequent analysis of the QoE computation results.

10. The method according to claim 9, further comprising:

receiving information associated with the one or more selected portions of the received data from one of the service provider or the content provider at the user device, wherein the one or more selected portions of the received data are video frames and, wherein the information associated with the one or more selected portions of the received data includes the video frames marked by one of the service provider or the content provider.

11. The method according to claim 9, wherein the one or more selected portions of the received data includes video frames and, wherein the video frames of the one or more selected portions of the received data are selected at the user device to be transmitted to one of the service provider or the content provider.

12. The method according to claim 11, further comprising:

transmitting, by the user device, identifiers of the video frames of the one or more selected portions of the received data to one of the service provider or the content provider such that reference frames for QoE computation are determined at one of the service provider or the content provider, wherein the video frames are selected based on one or more of an upload channel availability, a location of the user device, a mobility of the user device, a signal strength at the user device, available memory at the user device, and/or a processing capacity of the user device.

13. A server providing an audio/video transmission service with quality of experience (QoE) computation over a wireless network, the server comprising:

a communication module adapted to communicate wirelessly with a user device;

a memory adapted to store instructions; and

a processor coupled to the communication module and the memory, the processor adapted to;

transmit data to the user device, wherein the transmitted data is played back at the user device;

in response to a determination that sufficient bandwidth is available to receive one or more selected portions of the transmitted data from the user device, receive a largest amount of the one or more selected portions of the transmitted data that can be reliably received for real time QoE computation at one of a service provider or a content provider, wherein the one or more selected portions are selected on periodic intervals or based on a predefined algorithm on the user device;

in response to a determination that data upload and data download is shared through a same channel, receive the one or more selected portions of the transmitted data from the user device at a non-peak data transmission time period for the real time QoE computation at one of the service provider or the content provider such that additional bandwidth is released for upload purposes by downgrading a quality of transmissions;

perform the QoE computation based on the one or more selected portions of the transmitted data at one of the service provider or the content provider, wherein the transmitted data includes at least one of video data and/or audio data;

perform at least one of spatial and/or temporal alignment between an unimpaired reference frame of the one or more selected portions of the transmitted data and a corresponding frame of received portions of the transmitted data at one or more of the service provider or the content provider; and

transmit, by one of the service provider or the content provider, QoE computation results to a data store to subsequently analyze the QoE computation results and adjust QoE parameters based on the subsequent analysis of the QoE computation results.

14. The server according to claim 13, wherein the processor is further adapted to one or more of:

determine the one or more selected portions of the transmitted data at one of the service provider or the content provider;

transmit information associated with the one or more selected portions of the transmitted data from one of the service provider or the content provider to the user device;

use the one or more selected portions of the transmitted data to perform the QoE computation at one of the service provider or the content provider by comparing reference data portions of the transmitted data with the received portions of the transmitted data;

transmit information from one of the service provider or the content provider associated with the reference data portions of the transmitted data to be selected by marking video frames to be selected by the user device, wherein the reference data portions of the transmitted data are the video frames;

receive, from the user device, the one or more selected portions of the transmitted data through an upload channel; and

adjust one or more quality of service (QoS) parameters at one of the service provider or the content provider based on the QoE computation in real time.

15. The server according to claim 14, wherein the processor is further adapted to one or more of:

receive, at one of the service provider or the content provider, the one or more selected portions of the transmitted data from the user device during the non-peak data transmission time period;

store, at one of the service provider or the content provider, the received portions of the transmitted data for subsequent QoE computation; and

adjust, at one of the service provider or the content provider, one or more future quality of service (QoS) parameters based on the subsequent QoE computation.

16. The server according to claim 13, wherein the processor is further adapted to:

compare, at one of the service provider or the content provider, each pixel of an unimpaired reference frame of a portion of the transmitted data with each pixel of a corresponding frame of the received portions of the transmitted data.

17. The server according to claim 13, wherein the processor is further adapted to:

correlate, at one of the service provider or the content provider, results of the QoE computation with one or more of instantaneous QoS parameters, a location of the user device, a mobility of the user device, a signal strength at the user device, available memory at the user device, a processing capacity of the user device, and/or a load on communication network.

18. A computing device for receiving an audio/video transmission service with quality of experience (QoE) computation over a wireless network, the computing device comprising:

a communication module adapted to communicate wirelessly with a service provider;

a memory adapted to store instructions;

a processor coupled to the communication module and the memory, the processor adapted to;

receive data from the service provider to be played back at a user device;

determine one or more selected portions of the received data to be transmitted from the user device to the service provider for QoE computation;

capture the one or more selected portions of the received data to be transmitted to the service provider at the user device;

in response to a determination that sufficient bandwidth is available to transmit the one or more selected portions of the received data that includes at least one of video data and/or audio data from the user device through an upload channel, transmit a largest possible portion of the received data from the user device to the service provider that can be reliably transmitted for the QoE computation at one of the service provider or a content provider, wherein the one or more selected portions are selected on periodic intervals or based on a predefined algorithm;

in response to a determination that data upload and data download is shared through a same channel, transmit the one or more selected portions of the received data from the user device to the service provider at a non-peak data transmission time period for the QoE computation at one of the service provider or the content provider such that additional bandwidth is released for upload purposes by downgrading a quality of transmissions, wherein one of the service provider or the content provider is configured to;

perform at least one of spatial and/or temporal alignment between an unimpaired reference frame of the one or more selected portions of the received data received from one of the service provider and the content provider at the user device and a corresponding frame of one or more selected portions of the received data transmitted from the user device to one of the service provider or the content provider; and

transmit, QoE computation results to a data store to subsequently analyze the QoE computation results and adjust QoE parameters based on the subsequent analysis of the QoE computation results.

19. The computing device according to claim 18, wherein the processor is further adapted to:

receive information associated with the one or more selected portions of the received data selected from the service provider at the user device, wherein the one or more selected portions of the received data includes video frames and the information associated with the one or more selected portions of the received data that includes the video frames marked by the service provider.

20. The computing device according to claim 18, wherein the processor is further adapted to:

transmit identifiers of video frames from the user device to the service provider such that reference frames for QoE computation are determined at the service provider, and wherein the video frames are selected based on one or more of an upload channel availability, a location of the computing device, a mobility of the computing device, a signal strength at the computing device, available memory at the computing device, and/or a processing capacity of the computing device.