User attention-based adaptation of quality level to improve the management of real-time multi-media content delivery and distribution

• US 7,284,201 B2
• Filed: 09/20/2001
• Issued: 10/16/2007
• Est. Priority Date: 09/20/2001
• Status: Expired due to Fees

• Current Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Cohen-Solal, Eric
• Primary Examiner(s): Kincaid; Kristine
• Assistant Examiner(s): Vu; Thanh
• Application Number:
  US09/956,660
• Publication Number:
  US 20030052911A1
• Time in Patent Office: 2,217 Days
• Field of Search: 345/716, 345/738, 345/744, 345/863, 715/730, 715/716, 715/738, 715/744, 715/863, 715/763
• US Class Current: 715/738
• CPC Class Codes: H04L 47/24 Traffic characterised by specific attributes, e.g. priority or QoS
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/611 for multicast or broadcast systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/752 adapting media to network capabilities
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]
• CPC Class Codes: H04N 21/234345 the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment
• CPC Class Codes: H04N 21/234363 by altering the spatial resolution, e.g. for clients with a lower screen resolution
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/6582 Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number arrangements where receivers interact with the broadcast H04H20/38

Abstract

A method for transmitting a stream of multi-media content from provider server to a user device includes transmitting multi-media content from the provider server to the user device via a communication network and outputting the multi-media content from the user device to a user via an output on the user device such that the multi-media content is delivered from the provider server to the user in real-time. A degree of attention that the user directs to the output of the user device is continuously determined during the transmission and a parameter adjusting module at the provider server adjusts a parameter of the multi-media content in response to the degree of attention.

Claims

1. A method for distributing multi-media content in real-time, comprising the steps of:

(a) transmitting a stream of multi-media content from a provider server to a user device via a communication network and outputting the multi-media content on the user device in real-time;

(b) continuously determining a degree of attention that a user directs to the output of the user device during said step (a); and

(c) adjusting, at the provider server, a parameter of the multi-media content transmitted in said step (a) in response to the degree of attention determined in said step (b), wherein said step (c) comprises reducing a quality level of at least a portion of the multi-media content to reduce a bandwidth required for said step (a), wherein said step (b) further comprises sending a not attentive signal to the provider server when it is determined that the user'"'"'s attention is not directed to the output of the user device, and said step (c) comprises adjusting the parameter to a reduced quality level when the not attentive signal is present and adjusting the parameter to a normal quality level when the not attentive signal is not present, wherein said reducing is selectively performed to reduce quality of an audio signal while maintaining quality of a video signal, or to reduce quality of said video signal while maintaining said quality of said audio signal.

2. The method of claim 1, wherein said reducing is selectively performed based on a mute state of a television.

3. A method for distributing multi-media content in real-time, comprising the steps of:

(a) transmitting a stream of multi-media content from a provider server to a user device via a communication network and outputting the multi-media content on the user device in real-time;

(b) continuously determining a degree of attention that a user directs to the output of the user device during said step (a), said determining comprising measuring a distance between the user and the user device; and

(c) adjusting, at the provider server, a quality level of the multi-media content transmitted in said step (a) in response to the degree of attention determined in said step (b), wherein said quality level is reduced as said distance increases.

4. The method of claim 3, wherein the reduction of quality level with distance is gradual.

5. A method for distributing multi-media content in real-time, comprising the steps of:

(a) transmitting a stream of multi-media content from a provider server to a user device via a communication network and outputting the multi-media content on the user device in real-time at a normal quality level;

(b) continuously determining a degree of attention that a user directs to the output of the user device during said step (a); and

(c) adjusting, at the provider server, a parameter of the multi-media content transmitted in said step (a) in response to the degree of attention determined in said step (b), wherein said step (c) comprises reducing a quality level of at least a portion of the multi-media content to reduce a bandwidth required for said step (a), wherein said step (b) further comprises sending a not attentive signal to the provider server when it is determined that the user'"'"'s attention is not directed to the output of the user device, and said step (c) comprises adjusting the parameter to a reduced quality level when the not attentive signal is present and adjusting the parameter to the normal quality level when the not attentive signal is not present, wherein said reducing is selectively performed to reduce quality of an audio signal while maintaining quality of a video signal, or to reduce quality of said video signal while maintaining said quality of said audio signal.

6. The method of claim 5, wherein the determining the degree of user attention in step (b) comprises at least one of detecting skin tone percentage inside the face of the user and observing a gait of the user.

7. The method of claim 5, wherein the determining the degree of user attention in step (b) comprises at least one of applying a 3-D model to perform a fitting operation, learning 2-D facing and not facing models, and learning 3-D facing and not-facing models.

8. The method of claim 5, wherein said step (c) comprises reducing a quality level of at least a portion of one of an audio content and a video content of the multi-media content.

9. The method of claim 8, wherein said step (b) comprises determining areas on an output display where a user'"'"'s attention is directed and defining all other areas as non-attentive areas and the reduced quality level of at least a portion of one of an audio content and a video content is a reduced quality of the video content of the multi-media content in the non-attentive areas of the output display.

10. The method of claim 5, wherein the user device has an input for receiving a user command and the method further comprises the step of adjusting, at the provider server, a parameter of the multi-media content transmitted in said step (a) in response to a user command entered at the input.

11. The method of claim 5, wherein in said step (b) determining the degree of user attention comprises determining one of a distance of the user from the user device and a body orientation of the user.

12. The method of claim 5, wherein said reducing is from a normal quality level, and wherein said step (c) comprises, to keep the user from noticing change in said quality level, immediately resuming transmission at said normal quality level upon detecting that said user, after not being attentive to said output, has resumed being attentive to said output.

13. A system for distributing multi-media content over a network in real-time, comprising:

a plurality of detectors, each detector being configured for determining a degree of attention of a respective user to a respective user device, the user device being connected to said network for receiving a respective stream of multi-media content from a provider server and outputting the multi-media content to the respective user in real-time, the provider server being connected to the network for transmitting the stream of multi-media content in real-time to the respective user device at a normal quality level; and

a parameter adjusting module at said provider server for adjusting a parameter of the multi-media content being transmitted in response to the degree of attention determined by said detector, wherein said parameter adjusting module reduces a quality level of at least a portion of said stream of multi-media content being transmitted to said user device in response to a reduced degree of attention determined by said detector such that a bandwidth required for transmitting said stream of multi-media content is reduced to a lower quality level than the normal quality level, wherein said reducing is selectively performed to reduce quality of an audio signal while maintaining quality of a video signal, or to reduce quality of said video signal while maintaining said quality of said audio signal.

14. The system of claim 13, wherein said detector is operatively arranged for continually outputting a not attentive signal while it is determined that the user'"'"'s attention is not directed to at least a portion of the output of the user device, so that lack of said not attentive signal immediately indicates attention by said respective user.

15. The system of claim 13, wherein said user device comprises an input for receiving a user command and said parameter adjusting module is arranged for adjusting a parameter of the multi-media content being transmitted in response to the user command.

16. The system of claim 13, wherein said detector detects a distance of said respective user from said respective user device.

17. A system for distributing multi-media content over a network in real-time, comprising:

a detector for determining a degree of attention of the user to a user device, the user device being connectable to a network for receiving a stream of multi-media content from a provider server and outputting the multi-media content to a user in real-time, the provider server being connected to the network for transmitting the stream of multi-media content in real-time to the user device at a normal quality level; and

a parameter adjusting module at said provider server for adjusting a parameter of the multi-media content being transmitted in response to the degree of attention determined by said detector, wherein said adjusting comprises;

a) reducing a quality level of at least a portion of the output of the user device, from the normal quality level, wherein said reducing is selectively performed to reduce quality of an audio signal while maintaining quality of a video signal, or to reduce quality of said video signal while maintaining said quality of said audio signal; and

b) to keep the user from noticing change in said quality level, immediately resuming transmission at said normal quality level upon detecting that said user, after not being attentive to said output, has resumed being attentive to said output.

18. A system for distributing multi-media content over a network in real-time, comprising:

a detector for determining a degree of attention of the user to a user device, said determining comprising measuring a distance between the user and the user device, the user device being connectable to a network for receiving a stream of multi-media content from a provider server and outputting the multi-media content to a user in real-time at a normal quality level, the provider server being connected to the network for transmitting the stream of multi-media content in real-time to the user device; and

a parameter adjusting module at said provider server for adjusting a quality level of the multi-media content being transmitted in response to the degree of attention determined by said detector, wherein said user device has a screensaver mode and said parameter adjustment module is arranged for adjusting said quality level of the multi-media content to a lower quality level when the user is in the screensaver mode, said adjusting selectively performed to reduce quality of an audio signal while maintaining quality of a video signal, or to reduce quality of said video signal while maintaining said quality of said audio signal.