User attention-based adaptation of quality level to improve the management of real-time multi-media content delivery and distribution

• US 20030052911A1
• Filed: 09/20/2001
• Published: 03/20/2003
• Est. Priority Date: 09/20/2001
• Status: Active Grant

• Current Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Cohen-solal, Eric
• Granted Patent: US 7,284,201 B2
• Time in Patent Office: Days
• US Class Current: 345/738
• CPC Class Codes: H04L 47/24 Traffic characterised by specific attributes, e.g. priority or QoS
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/611 for multicast or broadcast systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/752 adapting media to network capabilities
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]
• CPC Class Codes: H04N 21/234345 the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment
• CPC Class Codes: H04N 21/234363 by altering the spatial resolution, e.g. for clients with a lower screen resolution
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/6582 Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number arrangements where receivers interact with the broadcast H04H20/38

Abstract

A method for transmitting a stream of multi-media content from provider server to a user device includes transmitting multi-media content from the provider server to the user device via a communication network and outputting the multi-media content from the user device to a user via an output on the user device such that the multi-media content is delivered from the provider server to the user in real-time. A degree of attention that the user directs to the output of the user device is continuously determined during the transmission and a parameter adjusting module at the provider server adjusts a parameter of the multi-media content in response to the degree of attention.

Claims

1. A method for distributing multi-media content in real-time, comprising the steps of:

(a) transmitting a stream of multi-media content from a provider server to a user device via a communication network and outputting the multi-media content on the user device in real-time;

(b) continuously determining a degree of attention that a user directs to the output of the user device during said step (a); and

(c) adjusting, at the provider server, a parameter of the multi-media content transmitted in said step (a) in response to the degree of attention determined in said step (b).

2. The method of claim 1, wherein in said step (b) determining the degree of user attention comprises determining one of a head orientation and a gaze of the user.

3. The method of claim 2, wherein said step of determining one of a head orientation and a gaze of the user includes determining head orientation by detecting a facial feature and at least one of using geometrical reasoning, applying a 3-D model to perform a fitting operation, learning 2-D facing and not facing models, learning 3-D facing and not-facing models, and detecting skin tone percentage inside the face.

4. The method of claim 1, wherein in said step (b) determining the degree of user attention comprises tracking an eye position of the user.

5. The method of claim 4, wherein said step of tracking an eye of the user includes determining a head orientation of a user by one of detecting a facial feature and at least one of using geometrical reasoning, applying a 3-D model to perform a fitting operation, learning 2-D facing and not facing models, learning 3-D facing an not-facing models, detecting skin tone percentage inside the face;

and

detecting the eye position by detecting eye features and using geometrical reasoning and performing learning techniques in a calibration phase to adapt to the user.

6. The method of claim 1, wherein said step (b) further comprises sending a not attentive signal to the provider server when it is determined that the user'"'"'s attention is not directed to the output of the user device.

7. The method of claim 6, wherein said not attentive signal comprises a one bit digital signal.

8. The method of claim 6, wherein said not attentive signal comprises a digital signal with more than one bit.

9. The method of claim 1, wherein said step (c) comprises reducing a quality level of at least a portion of the multi-media content to reduce a bandwidth required for said step (a).

10. The method of claim 9, wherein said step (c) comprises reducing a quality level of at least a portion of one of an audio content and a video content of the multi-media content.

11. The method of claim 10, wherein said step (b) comprises determining areas on an output display where a user'"'"'s attention is directed and defining all other areas as non-attentive areas and the reduced quality level of at least a portion of one of an audio content and a video content is a reduced quality of the video content of the multi-media content in the non-attentive areas of the output display.

12. The method of claim 1, wherein said step (b) further comprises sending a not attentive signal to the provider server when it is determined that the user'"'"'s attention is not directed to the output of the user device, and said step (c) comprises adjusting the parameter to a reduced quality level when the not attentive signal is present and adjusting the parameter to a normal quality level when the not attentive signal is not present.

13. The method of claim 1, wherein the user device has a screensaver mode and the method further comprises the step of adjusting, at the provider server, a parameter of the multi-media content transmitted in said step (a) when the user device is in the screensaver mode.

14. The method of claim 1, wherein the user device has an input for receiving a user command and the method further comprises the step of adjusting, at the provider server, a parameter of the multi-media content transmitted in said step (a) in response to a user command entered at the input.

15. The method of claim 1, wherein in said step (b) determining the degree of user attention comprises determining one of a presence of the user in an area in front of the user device, a distance of the user from the user device, and a body orientation of the user.

16. A system for distributing multi-media content over a network in real-time, comprising:

a detector for determining a degree of attention of the user to a user device, the user device being connectable to a network for receiving a stream of multi-media content form a provider server and outputting the multi-media content to a user in real-time, the provider server being connected to the network for transmitting the stream of multi-media content in real-time to the user device; and

a parameter adjusting module at said provider server for adjusting a parameter of the multi-media content being transmitted in response to the degree of attention determined by said detector.

17. The system of claim 16, wherein said detector comprises at least one of a camera and an infrared detector arranged for detecting one of an orientation and a gaze of the user.

18. The system of claim 16, wherein said detector comprises at least one of a camera and an infrared detector arranged for tracking an eye position of the user.

19. The system of claim 18, wherein the user device includes a display and said detector determines an attention area on said display at which a user'"'"'s attention is directed and defines the other areas of said display as non-attentive areas.

20. The system of claim 19, wherein said multi-media content comprises a video content and said parameter adjusting module reduces a quality of a video content in said non-attentive areas of said display, thereby reducing a bandwidth required for transmitting said stream of multi-media content.

21. The system of claim 16, wherein said parameter adjusting module reduces a quality level of at least a portion of said stream of multi-media content being transmitted to said user device in response to a reduced degree of attention determined by said detector such that a bandwidth required for transmitting said stream of multi-media content is reduced.

22. The system of claim 21, wherein said multi-media content comprises a video content and an audio content and said parameter adjusting module reduces a quality level of a portion of at least one of said video content and said audio content in response to the reduced degree of attention determined by said detector.

23. The system of claim 16, wherein said detector is operatively arranged for outputting a not attentive signal when it is determined that the user'"'"'s attention is not directed to at least a portion of the output of the user device.

24. The system of claim 23, wherein said parameter adjusting module is operatively arranged for adjusting the parameter of at least a portion of the multi-media content by the user device to a reduced quality level when the not attentive signal is present and adjusting the parameter to a normal quality level when the not attentive signal is not present.

25. The system of claim 16, wherein said user device has a screensaver mode and said parameter adjustment module is arranged for adjusting the parameter of the multi-media content when the user is in the screensaver mode.

26. The system of claim 16, wherein said user device comprises an input for receiving a user command and said parameter adjusting module is arranged for adjusting a parameter of the multi-media content being transmitted in response to the user command.

27. The system of claim 16, wherein said detector detects at least one of a presence of a user in an area in front of the user device, a distance of the user from the user device and an orientation of the user'"'"'s body.