Estimating engagement of consumers of presented content

• US 8,943,526 B2
• Filed: 04/19/2013
• Issued: 01/27/2015
• Est. Priority Date: 12/02/2011
• Status: Active Grant

• Current Assignee: Zhigu Holdings Limited
• Original Assignee: Microsoft Corporation
• Inventors: Rivera, Javier Hernandez, Liu, Zicheng, Hulten, Geoff, Conrad, Michael, Krum, Kyle, DeBarr, David, Zhang, Zhengyou
• Primary Examiner(s): NGUYEN BA, HOANG VU A
• Application Number:
  US13/866,699
• Publication Number:
  US 20130232515A1
• Time in Patent Office: 648 Days
• Field of Search: 725/10, 725/12, 705/14.41, 705/14.53, 705/14.66, 705/14.67
• US Class Current: 725/10
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0242 Determining effectiveness of advertisements
• CPC Class Codes: G06Q 30/0255 based on user history
• CPC Class Codes: G06Q 30/0269 based on user profile or attribute
• CPC Class Codes: G06Q 30/0271 Personalized advertisement
• CPC Class Codes: H04N 21/44213 Monitoring of end-user related data arrangements for monitoring the users' behaviour or opinions in broadcast systems H04H60/33
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/4532 involving end-user characteristics, e.g. viewer profile, preferences monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/4667 Processing of monitored end-user data, e.g. trend analysis based on the log file of viewer selections

Abstract

Technologies described herein relate to estimating engagement of a person with respect to content being presented to the person. A sensor outputs a stream of data relating to the person as the person is consuming the content. At least one feature is extracted from the stream of data, and a level of engagement of the person is estimated based at least in part upon the at least one feature. A computing function is performed based upon the estimated level of engagement of the person.

Claims

1. A computer-executable method, comprising:

receiving a stream of data output by a sensor, the stream of data relating to a person who is being presented with viewable content;

extracting a feature of the person from the stream of data;

computing a value that is indicative of engagement of the person with respect to the viewable content as the viewable content is being presented to the person, the computing of the value based at least in part upon the feature of the person extracted from the stream of data, wherein the computing of the value that is indicative of the engagement of the person comprises;

analyzing the feature of the person over a predefined time window, wherein the feature is subject to alteration; and

computing the value that is indicative of the engagement of the person based upon the analyzing of the feature of the person over the predefined time window; and

responsive to computing the value, performing at least one processing function based at least in part upon the value that is indicative of the engagement of the person, the performing of the at least one processing function undertaken by a computer processor.

2. The computer-executable method of claim 1, wherein the stream of data is a video stream output by a video camera, and wherein the feature of the person extracted from the video stream is a facial feature.

3. The computer-executable method of claim 1, wherein the stream of data is a video stream output by a video camera, and wherein the feature of the person extracted from the video stream is a pose of the head of the person.

4. The computer-executable method of claim 1, wherein the viewable content is one of a movie, a television program, a video, a song, or an advertisement.

5. The computer-executable method of claim 1, wherein the at least one processing function comprises:

selecting an advertisement from a plurality of potential advertisements based upon the value that is indicative of the engagement of the person; and

causing the advertisement selected from the plurality of potential advertisements to be presented to the person.

6. The computer-executable method of claim 1, wherein computing the value that is indicative of the engagement of the person comprises classifying the engagement of the person into one of a plurality of predefined levels of engagement.

7. The computer-executable method of claim 1 configured for execution in a video game console.

8. The computer-executable method of claim 1, wherein the at least one processing function comprises outputting a recommendation to the person as to other content predicted to be of interest to the person.

9. The computer-executable method of claim 1, wherein the at least one processing function comprises determining an instance in time to present an advertisement to the person.

10. The computer-executable method of claim 1, wherein the viewable content is an advertisement, and wherein the at least one processing function comprises selecting a price to charge an advertiser of the advertisement from amongst a plurality of potential prices based at least in part upon the value that is indicative of the engagement of the person with respect to the advertisement.

11. The computer-executable method of claim 1, wherein the feature extracted from the stream of data relates to a skeletal pose of the person.

12. A computing device, comprising:

a receiver component that is configured to receive a stream of data from a sensor, the stream of data relating to a person being presented with content;

a feature extractor component that is configured to extract a feature of the person from the stream of data in real-time;

an engagement estimator component that is configured to estimate a level of engagement of the person with respect to the content being presented to the person based at least in part upon the feature extracted from the stream of data by the feature extractor component over a predefined window of time, wherein the computing device is configured to perform at least one computing function based upon the level of engagement estimated by the engagement estimator component.

13. The computing device of claim 12 being a gaming console, and the sensor being at least one of a video camera or a depth sensor.

14. The computing device of claim 12, further comprising a recommender component that is configured to perform the at least one computing function, the at least one computing function comprising output of a recommendation as to other content predicted to be of interest to the person based upon the level of engagement of the person with respect to the content being presented to the person estimated by the engagement estimator component.

15. The computing device of claim 12, further comprising an advertisement selector component that is configured to perform the at least one computing function, the at least one computing function comprising presentment of an advertisement to the person based upon the level of engagement of the person with respect to the content being presented to the person estimated by the engagement estimator component.

16. The computing device of claim 12, wherein the sensor is a video camera, the feature extractor component configured to extract, in real-time, a plurality of features, the plurality of features comprising:

face distances, head pose, and head roll of the person in video data captured by the video camera, and the engagement estimator component configured to estimate the level of engagement of the person based upon the plurality of features.

17. The computing device of claim 12, wherein the content being presented to the person is one of a television program, a film, or an advertisement.

18. The computing device of claim 12, wherein the engagement estimator component is configured to output real-time feedback as to the level of engagement of the person to a provider of the content, the content being a speech or lecture.

20. The computing device of claim 12, the engagement estimator component further configured to estimate the level of engagement of the person with respect to the content being presented to the person based upon the content being presented to the person.

19. A computer-readable storage medium comprising instructions that, when executed by a processor, cause the processor to perform acts comprising:

receiving, from a video camera, a video feed directed at a person while the person is viewing media content;

extracting, in real-time, a plurality of features of the person from the video feed, the plurality of features comprising face distances identified in the video feed;

estimating a level of engagement of the person with respect to the media content based upon the plurality of features extracted from the video feed over a predefined window of time, the level of engagement being one of a plurality of predefined levels of engagement; and

updating the media content being viewed by the person based upon the level of engagement of the person estimated with respect to the media content.