Apparatus, systems and methods for generating an emotional-based content recommendation list

• US 10,764,641 B2
• Filed: 09/11/2018
• Issued: 09/01/2020
• Est. Priority Date: 12/06/2017
• Status: Active Grant

• Current Assignee: Dish Technologies LLC (Dish Network Corporation)
• Original Assignee: Dish Technologies LLC (Dish Network Corporation)
• Inventors: Newell, Nicholas, Subramanian, Prakash
• Primary Examiner(s): Gee, Alexander
• Application Number:
  US16/127,814
• Publication Number:
  US 20190174190A1
• Time in Patent Office: 721 Days
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/374 Detecting the frequency distribution of signals, e.g. detecting delta, theta, alpha, beta or gamma waves
• CPC Class Codes: A61B 5/378 Visual stimuli
• CPC Class Codes: A61B 5/6803 Head-worn items, e.g. helmets, masks, headphones or goggles
• CPC Class Codes: H04N 21/42201 biosensors, e.g. heat sensor for presence detection, EEG sensors or any limb activity sensors worn by the user input arrangements for interaction with the human body based on nervous system activity detection G06F3/015
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/4532 involving end-user characteristics, e.g. viewer profile, preferences monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/4667 Processing of monitored end-user data, e.g. trend analysis based on the log file of viewer selections
• CPC Class Codes: H04N 21/8405 represented by keywords

Abstract

Media content recommendation systems and methods are operable to recommend one or more media content events to a user based on identified changes in the user'"'"'s emotional state during a life event that is experienced by the user.

Claims

1. A method for recommending media content to a user based on emotions of the user, the method comprising:

receiving Electroencephalography (EEG) information corresponding to a plurality of brain wave lines of a user;

receiving image information captured by an image capture device that is oriented in a direction of a visual field of view of the user while the EEG information is acquired, wherein the EEG information and the image information are concurrently acquired;

identifying an emotional state change of the user based on the received EEG information;

identifying a candidate object in the captured image information, wherein the candidate object corresponds to a physical object seen by the user at a time when the emotional state change of the user was identified;

defining an emotional statement based on the identified emotional state change of the user and the identified candidate object, wherein the defined emotional statement is a textual statement that describes the identified candidate object, and wherein the emotional statement comprises a keyword that describes the identified candidate object;

comparing the keyword of the emotional statement with a plurality of media content event keywords, wherein each of the media content event keywords are associated with one or more of a plurality of media content events; and

identifying one of a plurality of media content events as a recommended media content event in response to the media content event keyword matching the keyword of the emotional statement.

2. The method of claim 1, wherein after identifying the emotional state change of the user based on the received EEG information, the method further comprising:

determining that an emotional state of the user is anticipation when the emotional state change of the user indicates that the user has an emotional feeling of excitement about an event that is going to happen, or is likely to happen, in the near future as a result of experiencing a first type of life event;

determining that the emotional state of the user is surprise when the emotional state change of the user indicates that the user has an emotional feeling of wonder, astonishment, or amazement as a result of experiencing a second type of life event that was unanticipated or that occurred unexpectedly;

determining that the emotional state of the user is joy when the emotional state change of the user indicates that the user has an emotional feeling of well-being, success, good fortune, or by the prospect of possessing what one desires as a result of experiencing a third type of life event;

determining that the emotional state of the user is sadness when the emotional state change of the user indicates that the user has an emotional feeling characterized by feelings of disadvantage, loss, despair, grief, helplessness, disappointment and sorrow as a result of experiencing a fourth type of life event;

determining that the emotional state of the user is disgust when the emotional state change of the user indicates that the user has an emotional feeling of revulsion or profound disapproval aroused by something unpleasant or offensive;

determining that the emotional state of the user is trust when the emotional state change of the user indicates that the user has an emotional feeling that someone is good and honest and will not harm the user, or that someone or something may be relied upon as a result of experiencing a sixth type of life event;

determining that the emotional state of the user is anger when the emotional state change of the user indicates that the user has an emotional feeling characterized by a strong feeling of annoyance, displeasure, or hostility as a result of experiencing a seventh type of life event;

determining that the emotional state of the user is fear when the emotional state change of the user indicates that the user has an emotional feeling of an unpleasant anticipation or awareness of danger, pain, or harm as a result of experiencing an eighth type of life event;

generating a media content event recommendation,

wherein the recommended media content event is indicated to the user as a media content event that the user is likely to enjoy in response to determining that the emotional state of the user is anticipation, joy, surprise or trust,

wherein the recommended media content event is indicated to the user as a media content event that the user is likely to dislike in response to determining that the emotional state of the user is sadness, disgust, anger or fear, and

wherein the recommended media content event is indicated to the user as a media content event that the user is likely to be disinterested in when determining that the emotional state of the user is not one of anticipation, surprise, joy, sadness, disgust, trust, anger, or fear; and

presenting the media content event recommendation to the user.

3. The method of claim 2, further comprising:

generating a user query that asks the user to specify their emotional state during the emotional state change of the user;

comparing the user'"'"'s specified emotional state to the determined emotional state;

confirming that the determined emotional state is correctly determined when the determined emotional state is the same as the user'"'"'s specified emotional state; and

changing the determined emotional state to the user'"'"'s specified emotional state when the determined emotional state is different from the user'"'"'s specified emotional state.

4. The method of claim 1, wherein the candidate object identified in the captured image information is a first candidate object, wherein a second candidate object is also identified in the captured image information, and wherein prior to identifying one of the plurality of media content events as the recommended media content event, the method further comprising:

presenting a first object image to the user, wherein the first object image includes the first candidate object;

presenting a second object image to the user, wherein the second object image includes the second candidate object; and

receiving a response from the user indicating that viewing one of the first candidate object or the second candidate object precipitated the emotional state change of the user,

wherein identifying one of the plurality of media content events as the recommended media content event is based on the indicated one of the first candidate object or the second candidate object.

5. The method of claim 3, further comprising:

generating a user query that asks the user to select one of the first candidate object or the second candidate object as precipitating the change in the user'"'"'s emotional state during the emotional state change of the user,

wherein the response from the user is a selection of one of the first candidate object or the second candidate object.

6. The method of claim 1, wherein after the candidate object is identified in the captured image information and before the recommended media content event is identified, the method further comprising:

presenting an image of the candidate object to the user;

generating a user query that asks the user to specify an emotional state that they experienced during the identified emotional state change of the user; and

receiving a user response that specified their emotional state that they experienced during the emotional state change,

wherein one of the plurality of media content events is identified as the recommended media content event based on the user specified emotional state that they experienced during the emotional state change.

7. The method of claim 1, wherein after the candidate object is identified in the captured image information and before the recommended media content event is identified, the method further comprising:

presenting an image of the candidate object to the user;

generating a user query that asks the user to confirm that the candidate object caused the identified emotional state change of the user; and

receiving a user response that confirms that the candidate object caused the identified emotional state change of the user, wherein the recommended media content event is identified based on the confirmed candidate object.

8. The method of claim 7, wherein the candidate object identified in the captured image information is a first candidate object, and wherein the received user response does not confirm that the candidate object caused the identified emotional state change of the user, the method further comprising:

identifying a second candidate object in the captured image information when the emotional state change of the user is identified;

generating a user query that asks the user to confirm that the second candidate object caused the identified emotional state change of the user; and

receiving a user response that confirms that the second candidate object caused the identified emotional state change of the user, wherein the recommended media content event is identified based on the confirmed second candidate object.

9. The method of claim 1, further comprising:

receiving eye orientation information acquired by an eye orientation sensor that senses eye orientation of the user while the image information is acquired by the image capture device, wherein the eye orientation information indicates an orientation of at least one eye of the user,

wherein identifying the candidate object in the captured image information is based on the orientation of the at least one eye of the user that indicates which one of a plurality of objects the user was looking at that were within the visual field of view of the user when the image information was acquired.

10. The method of claim 1, further comprising:

receiving audio information acquired by a microphone that includes sounds heard by the user while the image information is acquired by the image capture device; and

identifying a candidate sound in the audio information, wherein the candidate sound corresponds to a sound heard by the user during the emotional state change of the user,

wherein identifying one of the plurality of media content events as a recommended media content event is based on the identified candidate sound.

11. The method of claim 10, further comprising:

generating a user query that asks the user to confirm that the candidate sound precipitated the emotional state change of the user; and

receiving a user response that confirms that the sound caused the identified emotional state change of the user, wherein the recommended media content event is identified based on the confirmed candidate sound.

12. The method of claim 1, further comprising:

receiving audio information acquired by a microphone that includes sounds heard by the user while the image information is acquired by the image capture device;

identifying a candidate sound in the audio information, wherein the candidate sound corresponds to a sound heard by the user during the emotional state change of the user;

generating a user query that asks the user to select one of the candidate object or the candidate sound as precipitating the change in the user'"'"'s emotional state during the emotional state change of the user; and

receiving a user response from the user that is a selection of one of the candidate object or the candidate sound,

wherein identifying one of the plurality of media content events as a recommended media content event is based on the identified candidate object when the user selects the candidate object, and

wherein identifying one of the plurality of media content events as a recommended media content event is based on the identified candidate sound when the user selects the candidate sound.

13. The method of claim 1, wherein after the candidate object is identified in the captured image information and before the recommended media content event is identified, the method further comprising:

presenting an image of the candidate object to the user;

generating a user query that asks the user to specify if an emotional state that they experienced during the identified emotional state change of the user occurred during a real life experience or a content experience, wherein the real live experience occurs during a real life event experienced by the user, and wherein the content experience occurs while the user is viewing a media content event; and

receiving a user response that specifies one of the real life experience and the content experience, wherein the recommended media content event is identified based on the user specified one of the real life experience and the content experience.

14. The method of claim 1, wherein the EEG information includes first time information that defines time of acquisition of the EEG information, and wherein the image information includes second time information that defines time of acquisition of the image information by the image capture device, the method further comprising:

determining a first time from the first time information, wherein the first time corresponds to an onset of the emotional state change of the user; and

determining a second time from the second time information, wherein the second time is the same as the first time,

wherein the candidate object is identified from the image information that was acquired at the second time.

15. The method of claim 14, wherein determining the second time from the second time information further comprises:

adding a perception time duration to the first time from the first time information to determine the second time,

wherein the second time precedes the first time that corresponds to the onset of the emotional state change of the user by the perception time duration.

16. The method of claim 1, further comprising:

determining whether the user was experiencing a real life experience or a content experience during the emotional state change of the user, wherein the real live experience occurs during a real life event experienced by the user, and wherein the content experience occurs while the user is viewing a media content event; and

determining, in response to determining that the user was experiencing the content experience, an identifier of the media content event that the user was viewing,

wherein the recommended media content event is identified based on the determined identifier of the media content event that the user was viewing during the emotional state change of the user.

17. A device, comprising:

a transceiver or connector that communicatively couples an emotional state monitor (ESM) to the device,

wherein the ESM communicates Electroencephalography (EEG) information corresponding to a plurality of brain wave lines of a user being monitored by the ESM, and

wherein the ESM communicates image information captured by an image capture device of the ESM that is oriented in a direction of a visual field of view of the user while the EEG information is acquired, and

wherein the EEG information and the image information are concurrently acquired;

a memory that stores the EEG information and the image information; and

a processor system, wherein the processor system is configured to;

identify an emotional state change of the user based on the received EEG information;

identify a candidate object in the captured image information, wherein the candidate object corresponds to a physical object seen by the user at a time when the emotional state change of the user was identified;

define an emotional statement based on the identified emotional state change of the user and the identified candidate object, wherein the defined emotional statement is a textual statement that describes the identified candidate object, and wherein the emotional statement comprises a keyword that describes the identified candidate object;

retrieve a plurality of media content event keywords that are associated with one or more of a plurality of media content events;

compare the keyword of the emotional statement with the plurality of media content event keywords, wherein each of the media content event keywords are associated with one or more of a plurality of media content events; and

identify one of a plurality of media content events as a recommended media content event in response to the media content event keyword matching the keyword of the emotional statement.

18. The device of claim 17, wherein the processor system is further configured to:

receive eye orientation information acquired by an eye orientation sensor of the ESM, wherein the eye orientation sensor senses eye orientation of the user while the image information is acquired by the image capture device, wherein the eye orientation information indicates an orientation of at least one eye of the user,

wherein the candidate object in the captured image information is identified based on the orientation of the at least one eye of the user that indicates which one of a plurality of objects the user was looking at that were within the visual field of view of the user when the image information was acquired.

19. A method for recommending media content to a user based on emotions of the user, the method comprising:

receiving Electroencephalography (EEG) information corresponding to a plurality of brain wave lines of a user;

receiving image information captured by an image capture device that is oriented in a direction of a visual field of view of the user while the EEG information is acquired, wherein the EEG information and the image information are concurrently acquired;

identifying an emotional state change of the user based on the received EEG information;

identifying a candidate object in the captured image information, wherein the candidate object corresponds to a physical object seen by the user at a time when the emotional state change of the user was identified;

presenting an image of the candidate object to the user;

generating a user query that asks the user to confirm that the candidate object caused the identified emotional state change of the user;

receiving a user response that confirms that the candidate object caused the identified emotional state change of the user, wherein the recommended media content event is identified based on the confirmed candidate object; and

identifying one of the plurality of media content events as a recommended media content event based on the candidate object that was identified in the image information.

20. The method of claim 19, wherein the candidate object identified in the captured image information is a first candidate object, and wherein the received user response does not confirm that the candidate object caused the identified emotional state change of the user, the method further comprising:

identifying a second candidate object in the captured image information when the emotional state change of the user is identified;

generating a user query that asks the user to confirm that the second candidate object caused the identified emotional state change of the user; and

receiving a user response that confirms that the second candidate object caused the identified emotional state change of the user, wherein the recommended media content event is identified based on the confirmed second candidate object.