SUGGESTING EMOJI CHARACTERS BASED ON CURRENT CONTEXTUAL EMOTIONAL STATE OF USER

• US 20170083506A1
• Filed: 09/21/2015
• Published: 03/23/2017
• Est. Priority Date: 09/21/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Liu, Su, Rozner, Eric J., Sze, Chin Ngai, Wei, Yaoguang
• Granted Patent: US 9,665,567 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 2203/011 Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/10 Text processing natural language analysis G06F40/20; semantic analysis G06F40/30; processing or translation of natural language G06F40/40
• CPC Class Codes: G06F 40/166 Editing, e.g. inserting or deleting
• CPC Class Codes: G06F 40/279 Recognition of textual entities
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G06V 40/15 Biometric patterns based on physiological signals, e.g. heartbeat, blood flow
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: G10L 13/033 Voice editing, e.g. manipulating the voice of the synthesiser
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 25/63 for estimating an emotional state
• CPC Class Codes: H04M 1/72436 for text messaging, e.g. short messaging services [SMS] or e-mails
• CPC Class Codes: H04M 2250/12 including a sensor for measuring a physical value, e.g. temperature or motion telephones including GPS signal receivers H04M2250/10

Abstract

A current perceived emotional state of a user is determined. A semantic content of text inputted by the user can also be determined, as can an ongoing context in which the text inputted by the user pertains. A current contextual emotional state of the user is determined based on the text inputted by the user (such as based on the semantic content thereof) and based on the current perceived emotional state of the user. The current contextual emotional state can also be determined based on the ongoing context in which the text inputted by the user pertains. From emoji characters mapped to different contextual emotional states, one or more selected emoji characters are determined that are mapped to the current contextual emotional state of the user. The selected emoji characters are suggested to the user to add to the text inputted by the user.

Claims

1. A method comprising:

determining, by a computing device, a first emotional state of a user, as a current perceived emotional state of the user from information other than text inputted by the user on the computing device, by one or more of;

determining the current perceived emotional state from biometric information of the user detected by a biometric sensing device;

determining the current perceived emotional state from a facial image of the user detected by a camera device;

determining the current perceived emotional state from recorded audio of the user detected by an audio recording device;

determining, by the computing device, a second emotional state of the user, from a semantic content of the text inputted by the user;

determining, by the computing device, a current contextual emotional state of the user based on the text inputted by the user on the computing device and based on the current perceived emotional state of the user by;

when the first emotional state and the second emotional state are consistent with one another, ascertaining the current contextual emotional state of the user as a high degree of the first emotional state of the user;

when the first emotional state is inapposite, ascertaining the current contextual emotional state of the user as a baseline degree of the second emotional state;

when the second emotional state is inapposite, ascertaining the current contextual emotional state of the user as a baseline degree of the first emotional state;

when the first emotional state and the second emotional state are inconsistent with one another, ascertaining the current contextual emotional state of the user as irony;

from a plurality of emoji characters mapped to different contextual emotional states, determining, by the computing device, one or more selected emoji characters that are mapped to the current contextual emotional state of the user, each emoji character being a picture character or pictograph that is a single character with a unique code point value;

displaying, by the computing device, the selected emoji characters to the user as suggested emoji characters relevant to the current contextual emotion state of the user;

receiving, by the computing device, user selection of a particular selected emoji character from the selected emoji characters displayed to the user; and

adding the particular selected emoji character to the text inputted by the user.

9. The method of claim 1, further comprising:

determining an ongoing context in which the text inputted by the user pertains;

determining a third emotional state of the user from the ongoing context in which the text inputted by the user pertains,

wherein determining the current contextual emotional state of the user further comprises;

comparing the first emotional state, the second emotional state, and the third emotional state to one another; and

when the first emotional state, the second emotional state, and the third emotional state are consistent with one another, ascertaining the current contextual emotional state of the user as a highest degree of the current perceived emotional state of the user.

10. The method of claim 9, wherein determining the current contextual emotional state of the user further comprises:

when two of the first, second, and third emotional states are consistent with one another and a remaining of the first, second, and third emotional states is inapposite, ascertaining the current contextual emotional state of the user as a high degree of the two of the first, second, and third emotional states.

11. The method of claim 10, wherein determining the current contextual emotional state of the user further comprises:

when two of the first, second, and third emotional states are inapposite, ascertaining the current contextual emotional state of the user as a baseline degree of a remaining of the first, second, and third emotional states.

12. The method of claim 11, wherein determining the current contextual emotional state of the user comprises:

when at least one of the first, second, and third emotional states are inconsistent with at least one other of the first, second, and third emotional states, ascertaining the current contextual emotional state of the user as irony.

13. The method of claim 1, wherein the different contextual emotional states to which the emoji characters are mapped comprise multiple degrees of each of one or more base emotional states.

2. (canceled)

3. (canceled)

4. (canceled)

5. (canceled)

6. (canceled)

7. (canceled)

8. (canceled)

14. A computer program product comprising a memory having stored thereon program instructions executable by a computing device to cause the computing device to:

determine a first emotional state of a user, as a current perceived emotional state of the user from information other than text inputted by the user on the computing device, by one or more of, by one or more of;

determining the current perceived emotional state from biometric information of the user detected by a biometric sensing device;

determining the current perceived emotional state from a facial image of the user detected by a camera device;

determining the current perceived emotional state from recorded audio of the user detected by an audio recording device;

determine a second emotional state, from a semantic content of the text inputted by the user;

determine a current contextual emotional state of the user based on the semantic content of the text inputted by the user and based on the current perceived emotional state of the user by;

when the first emotional state and the second emotional state are consistent with one another, ascertaining the current contextual emotional state of the user as a high degree of the first emotional state of the user;

when the first emotional state is inapposite, ascertaining the current contextual emotional state of the user as a baseline degree of the second emotional state;

when the second emotional state is inapposite, ascertaining the current contextual emotional state of the user as a baseline degree of the first emotional state;

when the first emotional state and the second emotional state are inconsistent with one another, ascertaining the current contextual emotional state of the user as irony;

from a plurality of emoji characters mapped to different contextual emotional states, determine one or more selected emoji characters that are mapped to the current contextual emotional state of the user, each emoji character being a picture character or pictograph that is a single character with a unique code point value;

display the selected emoji characters to the user as suggested emoji characters relevant to the current contextual emotion state of the user;

receive user selection of a particular selected emoji character from the selected emoji characters displayed to the user; and

add the particular selected emoji character to the text inputted by the user.

15. (canceled)

16. (canceled)

17. (canceled)

18. A computing device comprising:

one or more of;

a biometric sensing device to detect biometric information of a user;

a camera device to detect a facial image of the user;

an audio recording device to detect recorded audio of the user;

a processor;

a memory; and

program instructions stored in the memory and executable by the processor to;

determine a first emotional state of the user, as a current perceived emotional state of a user from information other than text inputted by the user on the computing device, from one or more of the biometric information, the facial image, and the recorded audio;

determine semantic content of the text inputted by the user;

determine a second emotional state of the user, from the semantic content of the text inputted by the user;

determine an ongoing context in which the text inputted by the user pertains;

determine a third emotional state of the user, from the ongoing text in which the text inputted by the user pertains;

determine a current contextual emotional state of the user based on the ongoing context in which the text inputted by the user pertains, based on the semantic content of the text inputted by the user, and based on the current perceived emotional state of the user by;

comparing the first emotional state, the second emotional state, and the third emotional state to one another;

when the first emotional state, the second emotional state, and the third emotional state are consistent with one another, ascertaining the current contextual emotional state of the user as a highest degree of the current perceived emotional state of the user;

when two of the first, second, and third emotional states are consistent with one another and a remaining of the first, second, and third emotional states is inapposite, ascertaining the current contextual emotional state of the user as a high degree of the two of the first, second, and third emotional states;

when two of the first, second, and third emotional states are inapposite, ascertaining the current contextual emotional state of the user as a baseline degree of a remaining of the first, second, and third emotional states; and

when at least one of the first, second, and third emotional states are inconsistent with at least one other of the first, second, and third emotional states, ascertaining the current contextual emotional state of the user as irony;

from a plurality of emoji characters mapped to different contextual emotional states, determine one or more selected emoji characters that are mapped to the current contextual emotional state of the user, each emoji character being a picture character or pictograph that is a single character with a unique code point value; and

display the selected emoji characters to the user as suggested emoji characters relevant to the current contextual emotion state of the user;

receive user selection of a particular selected emoji characters displayed to the user; and

add the particular selected emoji character to the text inputted by the user.

19. (canceled)

20. (canceled)