Translating emotion to braille, emoticons and other special symbols

• US 20050069852A1
• Filed: 09/25/2003
• Published: 03/31/2005
• Est. Priority Date: 09/25/2003
• Status: Active Grant

• Current Assignee: International Business Machines Corporation, ServiceNow Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Janakiraman, Janani, Ullmannn, Cristi Nesbitt
• Granted Patent: US 7,607,097 B2
• Time in Patent Office: Days
• US Class Current: 434/236
• CPC Class Codes: G09B 21/003 using tactile presentation of the information, e.g. Braille displays
• CPC Class Codes: H04M 1/2474 Telephone terminals specially adapted for disabled people user interfaces for cordless or mobile telephones specially adapted for disabled users H04M1/72475; network based special services or facilities for hearing-impaired persons H04M3/42391; devices for conversing with the deaf-blind G09B21/04

Abstract

A system for incorporating emotional information in a communication stream from a first person to a second person, including an electronic image analyzer for determining an emotional component of a speaker or presenter using subsystems such as facial expression recognition, hand gesture recognition, body movement recognition, voice pitch analysis. A symbol generator generates one or more symbols such as emoticons, graphic symbols, or text modifications (bolding, underlining, etc.) corresponding to the emotional aspects of the presenter or speaker. The emotional symbols are merged with the audio or visual information from the speaker, and is presented to one or more recipients.

Claims

1. A computer-implemented method of communicating emotional aspects associated with a communication session from a first person to a second person, said method comprising the steps of:

receiving one or more emotional characteristic indicators from at least one emotional content analyzer system as the result of at least one emotional content analysis performed on said communication session from said first person;

identifying said first person and said second person;

determining an overall emotional state for said first person relative to said first person'"'"'s identity and said received emotional content analysis results;

generating one or more electronic symbols representing said overall emotion, said electronic symbols being interpretable by said second person; and

presenting said electronic symbols to said second person in association with said communication session.

2. The computer-implemented method as set forth in claim 1 wherein said step of receiving results of at least one emotional content analysis comprises receiving the results of a process selected from the group of a hand gesture recognizer, a body movement recognizer, a voice pitch analyzer, and a facial expression recognizer.

3. The computer-implemented method as set forth in claim 1 wherein said step of identifying said first person comprises accessing an electronic cultural profile for said first person to assist in determining said overall emotional state.

4. The computer-implemented method as set forth in claim 1 wherein said step of identifying said second person comprises accessing an electronic cultural profile for said second person to assist in determining appropriate electronic symbols to accurately represent said overall emotional state of said first person to said second person.

5. The computer-implemented method as set forth in claim 1 wherein said step of generating one or more electronic symbols corresponding to said overall emotional state comprises generating an electronic symbol selected from the list of a text-based emoticon, a graphical emoticon, a text highlight method, a text size change, a text underlining method, a text bolding method, one or more signals for a Telephone Terminal for the Deaf system, and a Braille code.

6. A computer readable media encoded with software for communicating emotional aspects associated with a communication session from a first person to a second person, said software causing a computer to perform the steps of:

receiving via a computer-readable communication means one or more emotional characteristic indicators from at least one emotional content analyzer system as the result of at least one emotional content analysis performed on said communication session from said first person;

identifying said first person and said second person;

determining an overall emotional state for said first person relative to said first person'"'"'s identity and said received emotional content analysis results;

generating one or more electronic symbols representing said overall emotion, said electronic symbols being interpretable by said second person; and

presenting on a user interface device said electronic symbols to said second person in association with said communication session.

7. The computer readable medium as set forth in claim 6 wherein said software for receiving results of at least one emotional content analysis comprises software for receiving the results of a process selected from the group of a hand gesture recognizer, a body movement recognizer, a voice pitch analyzer, and a facial expression recognizer.

8. The computer readable medium as set forth in claim 6 wherein said software for identifying said first person comprises software for accessing a electronic cultural profile for said first person to assist in determining said overall emotional state.

9. The computer readable medium as set forth in claim 6 wherein said software for identifying said second person comprises software for accessing an electronic cultural profile for said second person to assist in determining appropriate symbols to accurately represent said overall emotional state of said first person to said second person.

10. The computer readable medium as set forth in claim 6 wherein said software for generating one or more electronic symbols corresponding to said overall emotional state comprises software for generating an electronic symbol selected from the list of a text-based emoticon, a graphical emoticon, a text highlight method, a text size change, a text underlining method, a text bolding method, one or more signals for a Telephone Terminal for the Deaf system, and a Braille code.

11. A system for communicating emotional aspects of a communication session from a first person to a second person, said system comprising:

a recognizer input configured to receive results of at least one emotional content analyzer performed on said first person;

a set of user ID'"'"'s configured to identify said first person and said second person;

an emotional state analyzer adapted to determine an overall emotional state for said first person relative to said first person'"'"'s identity and said received emotional content analyzer results;

a symbol generator adapted to generate one or more symbols representing said overall emotion, said symbols being suitable for interpretation by said second person; and

a symbol merger for merging said generated symbols with said communication session for presentation to said second person.

12. The system as set forth in claim 11 wherein said recognizer input is further adapted to receive the results of a recognizer selected from the group of a hand gesture recognizer, a body movement recognizer, a voice pitch analyzer, and a facial expression recognizer.

13. The system as set forth in claim 11 wherein said user ID'"'"'s further comprises a cultural profile for said first person to assist in determining said overall emotional state.

14. The system as set forth in claim 11 wherein said user ID'"'"'s further comprise a cultural profile for said second person to assist in determining appropriate symbols to accurately represent said overall emotional state of said first person to said second person.

15. The system as set forth in claim 11 wherein said system generator is further adapted to generate a symbol selected from the list of a text-based emoticon, a graphical emoticon, a text highlight method, a text size change, a text underlining method, a text bolding method, one or more signals for a Telephone Terminal for the Deaf system, and a Braille code.

16. The system as set forth in claim 11 wherein said symbol merger is adapted to merge a communication session selected from the group of an electronic mail message, an online text chat, a video conference, a online classroom, a captioned television broadcast, a multimedia presentation, and an open captioned meeting.