IMAGE ANALYSIS FOR ATTENDANCE QUERY EVALUATION

• US 20160191995A1
• Filed: 03/04/2016
• Published: 06/30/2016
• Est. Priority Date: 09/30/2011
• Status: Abandoned Application

• Current Assignee: Affectiva, Inc. (Smart Eye AB)
• Original Assignee: Affectiva, Inc. (Smart Eye AB)
• Inventors: Mavadati, Seyedmohammad, el Kaliouby, Rana, Langeveld, Nicholas, McDuff, Daniel
• Application Number:
  US15/061,385
• Publication Number:
  US 20160191995A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: G06Q 30/0251 Targeted advertisements
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: G16H 20/70 relating to mental therapies, e.g. psychological therapy or autogenous training
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing
• CPC Class Codes: G16H 40/67 for remote operation
• CPC Class Codes: G16H 50/30 for calculating health indices; for individual health risk assessment
• CPC Class Codes: G16H 50/70 for mining of medical data, e.g. analysing previous cases of other patients
• CPC Class Codes: H04N 21/251 Learning process for intelligent management, e.g. learning user preferences for recommending movies details of learning user preferences for the retrieval of video data in a video database G06F16/739; computer systems using learning methods G06N3/08
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/4532 involving end-user characteristics, e.g. viewer profile, preferences monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/4667 Processing of monitored end-user data, e.g. trend analysis based on the log file of viewer selections
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02

Abstract

Facial evaluation is performed on one or more videos captured from an individual viewing a display. The images are evaluated to determine whether the display was viewed by the individual. The individual views a media presentation that includes incorporated tags and is rendered on the display. Based on the tags, video of the individual is captured and evaluated using a classifier. The evaluating includes determining whether the individual is in front of the screen, facing the screen, and gazing at the screen. An engagement score and emotional responses are determined for media and images provided on the display.

Claims

1. A computer-implemented method for analysis comprising:

receiving a plurality of images of an individual viewing an electronic display;

identifying a face of the individual wherein;

the identifying is based on a plurality of image classifiers;

the identifying occurs for at least one of the plurality of images; and

the plurality of image classifiers are used to perform head pose estimation; and

evaluating the plurality of images to determine that the electronic display was attended by the individual with the face.

2. The method of claim 1 wherein the receiving is in response to tagging of media rendered on the electronic display.

3. The method of claim 1 wherein being attended by the individual includes viewing of the electronic display.

4. The method of claim 3 wherein determination that the electronic display was attended is used in determining viewership.

5. The method of claim 3 wherein the electronic display renders an object and the viewing includes viewing the object.

6. The method of claim 1 wherein being attended by the individual includes determining viewability of digital media content from the electronic display.

7. The method of claim 6 wherein viewability includes evaluation of presence of digital media content and whether the digital media content is viewable by the individual.

8. The method of claim 7 wherein the evaluating the plurality of images includes scoring the digital media content.

9. The method of claim 8 wherein the scoring includes scoring for emotional reaction by the individual.

10. The method of claim 9 wherein the emotional reaction includes engagement.

11. The method of claim 8 wherein the digital media content includes an advertisement.

12. The method of claim 7 further comprising modifying the digital media content based on the viewability.

13. The method of claim 6 further comprising modifying the digital media content based on viewership.

14. The method of claim 1 further comprising performing eye gaze detection using the plurality of image classifiers.

15. The method of claim 1 wherein the evaluating the plurality of images is accomplished without eye tracking.

16. The method of claim 1 wherein the electronic display renders an advertisement and the advertisement has tagging incorporated.

17. The method of claim 16 further comprising invoking the evaluating based on the tagging that was incorporated.

20. The method of claim 1 further comprising determining an engagement score for the individual.

22. The method of claim 1 wherein an image classifier from the plurality of image classifiers is used to evaluate head pose for the individual.

23. The method of claim 1 wherein the evaluating is used as part of a viewership determination across a plurality of people.

24. The method of claim 1 further comprising evaluating emotional responses by the individual.

25. The method of claim 1 further comprising opting in by the individual for collection of the plurality of images.

26. The method of claim 25 wherein the opting in is persistent and was accomplished before an advertisement is rendered on the electronic display.

29. The method of claim 1 further comprising receiving a second plurality of images of a second individual viewing a second electronic display;

identifying a second face of the second individual wherein the identifying the second face is based on the plurality of image classifiers and wherein the identifying the second face occurs for at least one of the second plurality of images;

evaluating the second plurality of images to determine that the second electronic display was attended by the second individual; and

determining a viewership score based on the evaluating the plurality of images and the evaluating the second plurality of images.

30. The method of claim 1 wherein the evaluating comprises:

determining regions within the face of the individual and evaluating the regions for emotional content.

18-19. -19. (canceled)

21. (canceled)

27-28. -28. (canceled)

31. A computer program product embodied in a non-transitory computer readable medium for analysis, the computer program product comprising:

code for receiving a plurality of images of an individual viewing an electronic display;

code for identifying a face of the individual wherein;

the identifying is based on a plurality of image classifiers;

the identifying occurs for at least one of the plurality of images; and

the plurality of image classifiers are used to perform head pose estimation; and

code for evaluating the plurality of images to determine that the electronic display was attended by the individual.

32. A computer system for analysis comprising:

a memory which stores instructions;

one or more processors attached to the memory wherein the one or more processors, when executing the instructions which are stored, are configured to;

receive a plurality of images of an individual viewing an electronic display;

identify a face of the individual wherein;

identification is based on a plurality of image classifiers;

identification occurs for at least one of the plurality of images; and

the plurality of image classifiers are used to perform head pose estimation; and

evaluate the plurality of images to determine that the electronic display was attended by the individual with the face.