Determining Audience State or Interest Using Passive Sensor Data

• US 20150341692A1
• Filed: 07/31/2015
• Published: 11/26/2015
• Est. Priority Date: 12/09/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Conrad, Michael J., Hulten, Geoffrey J., Krum, Kyle J., Mendhro, Umaimah A., Remington, Darren B.
• Granted Patent: US 9,628,844 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: H04N 21/42201 biosensors, e.g. heat sensor for presence detection, EEG sensors or any limb activity sensors worn by the user input arrangements for interaction with the human body based on nervous system activity detection G06F3/015
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44213 Monitoring of end-user related data arrangements for monitoring the users' behaviour or opinions in broadcast systems H04H60/33
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/4667 Processing of monitored end-user data, e.g. trend analysis based on the log file of viewer selections

Abstract

This document describes techniques and apparatuses for determining an audience'"'"'s state or interest using passive sensor data. The techniques receive sensor data that measures an audience during presentation of a media program. The techniques, based on this sensor data, then determine a state or interest of the audience members during the media program.

Claims

1. A system comprising:

one or more processors; and

one or more memories storing instructions that, response to execution by the one or more processors, cause the one or more processors to perform operations comprising;

receiving a media program that includes multiple portions;

determining a media type for each of the multiple portions of the media program;

receiving sensor data for a person, the sensor data passively sensed during presentation of the media program to the person;

determining, based on the sensor data, a state of the person during each of the multiple portions of the media program;

determining, based on the sensor data, an overall interest level of the person during the multiple portions of the media program; and

providing the state of the person during each of the multiple portions of the media program and the overall interest level of the person during the multiple portions of the media program effective to enable measurement of a value of the media program, assessment of a potential value of a similar media program or portion thereof, control of presentation of the media program, or automatic rating, for the person, of the media program.

2. The system of claim 1, the operations further comprising determining, based on the sensor data, an identity of the person and providing the identity of the person.

3. The system of claim 1, the operations further comprising determining, based on the sensor data, a demographic of the person and providing the demographic of the person.

4. The system of claim 1, the operations further comprising determining, based on the media type and the state, an interest level of the person relative to the multiple portions of the media program.

5. The system of claim 1, wherein the sensor data includes head movement of the person, facial movement or orientation of the person, and high-amplitude audio from the person.

6. The system of claim 1, wherein the sensor data includes audio and the operation of determining the state determines, using speech recognition, the state to be a related talking state or an unrelated talking state.

7. The system of claim 1, wherein the operation of determining the state of the person determines a numerical probability for the state or multiple numerical probabilities for multiple states, respectively.

8. A computer-implemented method comprising:

receiving sensor data for an audience, the sensor data passively sensed during presentation of a media program to the audience;

determining, based on a media type describing a particular portion of the media program and the sensor data for the audience passively sensed during the presentation of the particular portion of the media program to the audience, an interest level of the audience to the particular portion of the media program;

producing a graph that indicates the determined interest level of the audience to the particular portion of the media program, the graph of the audience being usable to indicate a value of the media program, usable to assess a potential value of a similar media program or portion thereof, usable to control presentation of the media program, or usable to indicate a rating for the media program.

9. The computer-implemented method as described in claim 8, further comprising normalizing the determined interest level of the audience based on historical interest levels for a person in the audience.

10. The computer-implemented method as described in claim 9, wherein the graph indicates the determined and normalized interest level of the audience to the particular portion of the media program.

11. The computer-implemented method as described in claim 8, further comprising determining, based on the sensor data, an engagement of the audience to the particular portion of the media program, and where determining the interest level of the audience is based on the engagement of the audience and the media type.

12. The computer-implemented method as described in claim 8, further comprising determining, based on the sensor data, states of one or more persons in the audience during presentation of the particular portion of the media program, and wherein the determining the interest level of the audience is based on the states of the one or more persons and the media type.

13. The computer-implemented method as described in claim 8, wherein the sensor data for the audience includes multiple sensor readings over a time in which the media program is presented to the audience and determining the interest level of the audience determines multiple interest levels over the time.

14. A device comprising:

one or more processors; and

one or more computer-readable storage media storing instructions that, response to execution by the one or more processors, cause the one or more processors to perform operations comprising;

passively sensing or receiving sensor data for an audience, the sensor data passively sensed at time periods during which a media program is presented to the audience and from an audience-sensing that includes sensors to capture audio and movement of the audience;

determining, based on the sensor data, multiple states of at least one person in the audience for the time periods during which the media program is presented to the audience;

receiving one or more media types describing respective portions included in the media program;

determining, based on the one or more media types describing respective portions included in the media program and the multiple states of the person in the audience, multiple interest levels of the person for the time periods; and

providing, to a remote entity, the determined multiple interest levels to enable measurement of a value of the media program, assessment of a potential value of a similar media program or portion thereof, control of presentation of the media program, or automatic rating, for the person, of the media program.

15. The device of claim 14, wherein determining the multiple interest levels of the person for the time periods includes determining, for one of the time periods, at least one interest level based on a combination of the multiple states.

16. The device of claim 14, wherein the sensor data is received from an audience-sensing device local to the audience.

17. The device of claim 16, wherein the audience-sensing device is capable of sensing facial orientation and skeletal movement of the audience, and the sensor data includes facial orientation data and skeletal movement data for a person in the audience.

18. The device of claim 16, wherein the audience-sensing device includes a microphone capable of sensing audio generated by the audience and a location of the audio relative to persons in the audience.

19. The device of claim 16, the operations further comprising normalizing the determined multiple interest levels of the person based on historical interest levels for the person.

20. The device of claim 16, the operations further comprising determining, based on the sensor data, an identity of the person and providing the identity of the person.