Dynamic video streaming based on viewer activity

• US 10,129,312 B2
• Filed: 09/11/2014
• Issued: 11/13/2018
• Est. Priority Date: 09/11/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Spears, Stacey L., Wu, Haoyun
• Primary Examiner(s): Bengzon, Greg C
• Application Number:
  US14/483,976
• Publication Number:
  US 20160080448A1
• Time in Patent Office: 1,524 Days
• CPC Class Codes: H04L 65/613 for the control of the source by the destination control signals issued by the client directed to the server or network components specially adapted for selective content distribution H04N21/637
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: H04L 67/54 Presence management, e.g. monitoring or registration for receipt of user log-on information, or the connection status of the users
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44213 Monitoring of end-user related data arrangements for monitoring the users' behaviour or opinions in broadcast systems H04H60/33
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/6379 directed to encoder , e.g. for requesting a lower encoding rate

Abstract

In some examples, techniques and architectures for modifying resolution of video displayed on a display device include receiving information from a distance sensor and one or more image sensors, detecting an activity of a viewer of the display device based, at least in part, on the information received from the distance sensor and the one or more image sensors, and transmitting a request to a remote server to adjust bandwidth of media content streaming to the display device to a new bandwidth. The new bandwidth may be based, at least in part, on one or more of the activity of the viewer, size of the display device, or an ability of the viewer to resolve images displayed on the display device.

Claims

1. A method comprising:

testing, by a detector system, visual acuity of a viewer of a display device, operably connected to the detector system, to determine an ability of the viewer to resolve images displayed on the display device;

detecting, by the detector system, facial characteristics of the viewer;

storing, in a memory device associated with the display device, the facial characteristics and visual acuity information resulting from the testing in association with a user identity of the viewer;

upon the viewer returning to a vicinity of the display device, receiving distance information from a distance sensor, the distance information including a distance between the viewer at a viewing location and the display device to which media content is streaming;

determining, based on a recognition of the facial characteristics, the user identity of the viewer of the display device, the user identity being identified from a plurality of user identities associated with viewers that have previously viewed the display device;

accessing, from the memory device, the visual acuity information stored in association with the user identity;

generating a value for the viewer based at least in part on the distance information and the visual acuity information, the value indicative of an ability of the viewer to notice a reduction in resolution of the media content streaming to the display device;

determining a resolution based at least in part on the value; and

transmitting a request to a remote server to adjust bandwidth of the media content streaming to the display device to a new bandwidth, wherein the new bandwidth is based, at least in part, on a size of the display device and the resolution.

2. The method of claim 1, further comprising instructing the display device to display a prompt that allows the viewer to override the request to the remote server.

3. The method of claim 1, further comprising:

receiving image information from one or more image sensors; and

detecting an activity of the viewer of the display device based, at least in part, on the image information received from the one or more image sensors, wherein the value is further generated based at least in part on the activity.

4. The method of claim 3, further comprising:

comparing a history of behavior of the viewer with the activity of the viewer; and

delaying the request to the remote server based, at least in part, on the comparing.

5. The method of claim 3, further comprising:

detecting whether the viewer is wearing corrective lenses by using the one or more image sensors, wherein the new bandwidth is further based, at least in part, on results of the detecting.

6. The method of claim 1, wherein the value is generated using a mathematical relationship that comprises a plurality of adjustable factors to account for a plurality of different viewing parameters that includes at least the distance information, the visual acuity information, and user age information.

7. The method of claim 1, wherein the visual acuity information stored in association with the user identity is maintained in a look-up table.

8. The method of claim 1, wherein the testing the visual acuity of the viewer of the display device comprises displaying a Snellen-type visual test pattern for the viewer.

9. A system comprising:

a distance sensor;

a display device;

a detector operably connected to the display device;

one or more processing units; and

computer-readable media having instructions that, when executed by the one or more processing units, configure the one or more processing units to perform operations comprising;

testing, by the detector, visual acuity of a viewer of the display device to determine an ability of the viewer to resolve images displayed on the display device;

storing visual acuity information resulting from the testing in association with a user identity of the viewer;

detecting a distance between the display device and the viewer of the display device based, at least in part, on information from the distance sensor;

determining a user identity of the viewer, the user identity being identified from a plurality of user identities associated with viewers that have previously viewed the display device;

accessing visual acuity information associated with the user identity;

generating a value for the viewer based at least in part on the distance and the visual acuity information, the value indicative of an ability of the viewer to notice a reduction in resolution of media content streaming to the display device;

determining a resolution based at least in part on the value; and

transmitting a request to a remote server to adjust bandwidth of the media content streaming to the display device to a new bandwidth, wherein the new bandwidth is based, at least in part, on a size of the display device and the resolution.

10. The system of claim 9, wherein the operations further comprise:

detecting an activity of the viewer based, at least in part, on information from one or more image sensors, wherein the value is further generated based at least in part on the activity.

11. The system of claim 9, wherein the value is generated using a mathematical relationship that comprises a plurality of adjustable factors to account for a plurality of different viewing parameters that includes at least the distance, the visual acuity information, and user age information.

12. The system of claim 9, wherein the visual acuity information stored in association with the user identity are maintained in a look-up table.

13. A system comprising:

a display device;

a detector operably connected to the display device;

one or more processing units; and

one or more computer-readable media storing computer-executable instructions that, when executed by the one or more processing units, configure the one or more processing units to perform operations comprising;

testing, by the detector, visual acuity of a viewer of the display device to determine an ability of the viewer to resolve images displayed on the display device;

storing visual acuity information resulting from the testing in association with a user identity of the viewer for each of a plurality of user identities associated with viewers that have viewed the display device;

upon the viewer returning to a vicinity of the display device, using distance information from a distance sensor to determine a distance between the viewer of the display device and the display device;

determining the user identity of the viewer based on recognition data;

using image information from an image sensor to determine an activity of the viewer of the display device;

accessing the stored visual acuity information associated with the user identity;

generating a value for the viewer based at least in part on each of the distance, the visual acuity information, and the activity of the viewer, the value indicative of an ability of the viewer to notice a reduction in resolution of media content streaming to the display device;

determining a resolution based at least in part on the value; and

generating a request for transmission to a remote server, the request to adjust bandwidth of the media content streaming to the display device to a new bandwidth, wherein the new bandwidth is based, at least in part, on a size of the display device and the resolution.

14. The system of claim 13, wherein the operations further comprise:

comparing a history of behavior of the viewer with the activity of the viewer; and

modifying the request to the remote server based, at least in part, on the comparing.

15. The system of claim 13, wherein the value is generated using a mathematical relationship that comprises a plurality of adjustable factors to account for a plurality of different viewing parameters that includes each of the distance, the visual acuity information, and the activity of the viewer.

16. The system of claim 13, wherein the visual acuity information associated with the user identity is maintained in a look-up table.

17. The system of claim 13, wherein the activity of the viewer is associated with a category of activity, the operations further comprising:

determining a degree to which the viewer is engaged in the activity;

associating the degree to which the viewer is engaged in the activity with an attention value, wherein the value is generated based at least in part on the attention value.

18. The system of claim 17, wherein the category of activity is assigned to a range of attention values.

19. The system of claim 13, wherein the recognition data comprises facial characteristics and facial recognition is used to determine the user identity of the viewer.