IMAGE ANALYSIS FOR TWO-SIDED DATA HUB

• US 20180196432A1
• Filed: 03/12/2018
• Published: 07/12/2018
• Est. Priority Date: 06/07/2010
• Status: Active Grant

• Current Assignee: Affectiva, Inc. (Smart Eye AB)
• Original Assignee: Affectiva, Inc. (Smart Eye AB)
• Inventors: Krupat, Jason, el Kaliouby, Rana, Radice, Jason, Zijderveld, Gabriele, Cabot, Chilton Lyons
• Granted Patent: US 10,401,860 B2
• Time in Patent Office: Days
• CPC Class Codes: A61B 2576/02 specially adapted for a particular organ or body part
• CPC Class Codes: A61B 5/0013 Medical image data A61B1/00011, A61B6/56, A61B8/56 take precedence
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/1176 Recognition of faces
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: A61B 5/4803 Speech analysis specially adapted for diagnostic purposes
• CPC Class Codes: A61B 5/6893 Cars
• CPC Class Codes: B60W 2040/0872 Driver physiology
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2540/21 Voice
• CPC Class Codes: B60W 2540/22 Psychological state; Stress level or workload
• CPC Class Codes: B60W 2540/221 Physiology, e.g. weight, heartbeat, health or special needs
• CPC Class Codes: B60W 2540/223 Posture, e.g. hand, foot, or seat position, turned or inclined
• CPC Class Codes: B60W 2540/229 Attention level, e.g. attentive to driving, reading or sleeping
• CPC Class Codes: B60W 40/08 related to drivers or passengers
• CPC Class Codes: G05D 1/0291 Fleet control monitoring fleets in traffic control systems for road vehicles G08G1/127, G08G1/127
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06F 18/28 Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 10/764 : using classification, e.g. of video objects
• CPC Class Codes: G06V 10/772 : Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries
• CPC Class Codes: G06V 20/597 : Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/174 : Facial expression recognition
• CPC Class Codes: G08G 1/0962 : having an indicator mounted inside the vehicle, e.g. giving voice messages
• CPC Class Codes: G10L 15/1807 : using prosody or stress
• CPC Class Codes: G10L 15/1815 : Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/22 : Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 25/63 : for estimating an emotional state
• CPC Class Codes: G10L 25/90 : Pitch determination of speech signals

Abstract

Image analysis is performed for a two-sided data hub. Data reception on a first computing device is enabled by an individual and a content provider. Cognitive state data including facial data on the individual is collected on a second computing device. The cognitive state data is analyzed on a third computing device and the analysis is provided to the individual. The cognitive state data is evaluated and the evaluation is provided to the content provider. A mood dashboard is displayed to the individual based on the analyzing. The individual opts in to enable data reception for the individual. The content provider provides content via a website.

Claims

1. A computer-implemented method for image analysis comprising:

enabling, on a first computing device, data reception by an individual and a content provider;

collecting, on a second computing device, cognitive state data including facial data on the individual;

analyzing, on a third computing device, the cognitive state data for providing analysis of the cognitive state data to the individual; and

evaluating the cognitive state data for providing evaluation of the cognitive state data to the content provider.

2. The method of claim 1 further comprising displaying a mood dashboard to the individual based on the analyzing.

3. The method of claim 1 further comprising opting in by the individual to the enabling data reception for the individual.

4. The method of claim 1 wherein the collecting cognitive state data, including facial data on the individual, is accomplished using a webcam.

5. The method of claim 4 wherein the collecting cognitive state data further includes capturing physiological data.

6. The method of claim 4 wherein the webcam is integrated into a device displaying content from the content provider.

7. The method of claim 4 wherein the webcam is included in a different device than the device displaying content from the content provider.

8. The method of claim 1 wherein the enabling is accomplished using a cognitive reaction data hub.

9. The method of claim 8 wherein the cognitive reaction data hub is part of a two-sided marketplace.

10. The method of claim 8 wherein the evaluating includes generating a cognitive profile.

11. The method of claim 10 wherein the cognitive profile is used on the cognitive reaction data hub.

12. The method of claim 1 wherein the collecting of cognitive state data is accomplished within a vehicle and the individual is an occupant of the vehicle.

17. The method of claim 1 wherein the evaluating further comprises aggregating the cognitive state data with cognitive state data from other individuals.

18. The method of claim 17 wherein the cognitive state data from other individuals is based on demographics.

19. The method of claim 18 wherein the cognitive state data from other individuals based on demographics is used to provide a worldwide cognitive state map.

20. The method of claim 17 wherein the aggregating is based on time intervals.

21. The method of claim 1 wherein the analyzing further comprises comparing the cognitive state data with collected cognitive state data from other individuals.

22. The method of claim 21 wherein the comparing provides quantified self-information on cognitive states for the individual.

25. The method of claim 1 wherein the analyzing includes using an image classifier from a plurality of image classifiers to detect emotional content within the facial data for the individual.

31. The method of claim 1 wherein the analyzing the cognitive state data is used to enable the individual to track cognitive health.

32. The method of claim 1 wherein the evaluating is used to enable the content provider to analyze cognitive engagement.

33. The method of claim 1 further comprising augmenting the analyzing using audio data from the individual.

34. The method of claim 33 wherein the audio data includes voice data from the individual.

40. The method of claim 1 wherein the cognitive state data is indicative of drowsiness, fatigue, distraction, impairment, sadness, stress, happiness, anger, frustration, confusion, disappointment, hesitation, cognitive overload, focusing, engagement, attention, boredom, exploration, confidence, trust, delight, disgust, skepticism, doubt, satisfaction, excitement, laughter, calmness, curiosity, humor, depression, envy, sympathy, embarrassment, poignancy, or mirth.

13-16. -16. (canceled)

23-24. -24. (canceled)

26-30. -30. (canceled)

35-39. -39. (canceled)

41. A computer program product embodied in a non-transitory computer readable medium for image analysis, the computer program product comprising code which causes one or more processors to perform operations of:

enabling, on a first computing device, data reception by an individual and a content provider;

collecting, on a second computing device, cognitive state data including facial data on the individual;

analyzing, on a third computing device, the cognitive state data for providing analysis of the cognitive state data to the individual; and

evaluating the cognitive state data for providing evaluation of the cognitive state data to the content provider.

42. A computer system for image analysis comprising:

a memory which stores instructions;

one or more processors attached to the memory wherein the one or more processors, when executing the instructions which are stored, are configured to;

enable data reception by an individual and a content provider;

collect cognitive state data including facial data on the individual;

analyze the cognitive state data for providing analysis of the cognitive state data to the individual; and

evaluate the cognitive state data for providing evaluation of the cognitive state data to the content provider.