EMOTION RECOGNITION MESSAGE SYSTEM, MOBILE COMMUNICATION TERMINAL THEREFOR AND MESSAGE STORAGE SERVER THEREFOR

• US 20100003969A1
• Filed: 04/03/2009
• Published: 01/07/2010
• Est. Priority Date: 04/07/2008
• Status: Active Grant

• Current Assignee: NTT Docomo Incorporated (Nippon Telegraph and Telephone Corporation)
• Original Assignee: NTT Docomo Incorporated (Nippon Telegraph and Telephone Corporation)
• Inventors: Yabusaki, Masami, Isobe, Shin-ichi
• Granted Patent: US 8,285,257 B2
• Time in Patent Office: Days
• US Class Current: 455/412.1
• CPC Class Codes: H04M 2203/253 where a visual mode is used instead of a voice mode
• CPC Class Codes: H04M 3/42042 Notifying the called party of information on the calling party details within substation equipment H04M1/57, signalling details H04Q3/72
• CPC Class Codes: H04M 3/533 Voice mail systems

Abstract

A message storage server stores a message by speech or video transmitted from a mobile communication terminal and performs emotion recognition on the message to generate emotion data. A mobile communication terminal, upon receiving a confirmation request from the message storage server of a message which the mobile communication terminal has transmitted as a message for another mobile communication terminal, displays the emotion data assigned to the transmitted message on its display.

Claims

1. A message system having a message storage server and at least two mobile communication terminals,

each of the at least two mobile communication terminals comprising:

a transmitter-receiver that transmits data to, and receives data from, the message storage server;

a display that displays an image;

an input receiver that receives input of speech or video from a user; and

a controller that controls each of the transmitter-receiver, the display, and the input receiver,

wherein the controller that, in a case in which a message by speech or video for another mobile communication terminal is received by the input receiver, transmits the message to the message storage server via the transmitter-receiver;

the message storage server comprising;

a message receiver that receives, from the mobile communication terminal, the message for the another mobile communication terminal;

a storage device that stores the message received by the message receiver;

an emotion recognizer that performs emotion recognition on the message and that assigns emotion data indicating the recognized emotion to the message; and

a confirmation request transmitter that transmits, as a confirmation request, emotion data assigned by the emotion recognizer to the mobile communication terminal that has transmitted the message,

wherein the controller, upon receiving from the message storage server the confirmation request, displays an image corresponding to the received emotion data on the display.

2. A mobile communication terminal that connects to a mobile communication system having a message storage server and that communicates with another mobile communication terminal,

the mobile communication terminal comprising:

a transmitter-receiver that transmits data to, and receives data from, the message storage server for registering a message by speech or video received from the mobile communication terminal after performing emotion recognition on the received message and assigning emotion data indicating the recognized emotion to the received message;

a display that displays an image;

an input receiver that receives input of speech or video from a user; and

a controller that controls each of the transmitter-receiver, the display, and the input receiver,

wherein the controller, in a case in which a message by speech or video for another mobile communication terminal is received by the input receiver, transmits the message to the message storage server via the transmitter-receiver;

wherein the controller, upon receiving, as a confirmation request, emotion data for the transmitted message from the message storage server, displays an image corresponding to the received emotion data on the display; and

wherein the controller, upon receiving, via the input receiver, an input instruction indicating that the user agrees with a type of the emotion indicated by the image displayed on the display, transmits via the transmitter-receiver to the message storage server an approval response indicating approval to the emotion data.

3. A mobile communication terminal according to claim 2,

wherein the controller, upon receiving via the transmitter-receiver from the message storage server a registration notification that notifies registration of a message for the mobile communication terminal in the message storage server, transmits an access request for the registered message via the transmitter-receiver to the message storage server;

wherein the controller, upon receiving via the transmitter-receiver from the message storage server, screen data representing a selection screen that displays message information indicating attributes of the registered message, displays the selection screen represented by the screen data on the display, the selection screen displaying the message information of at least one message that can be downloaded by the mobile communication terminal, with the message information being displayed after being sorted according to a type of emotion indicated by the emotion data of the message; and

wherein the controller, when one of the at least one message is selected from the selection screen, transmits a download request for the selected message to the message storage server.

4. A mobile communication terminal according to claim 2,

wherein the controller, upon receiving via the transmitter-receiver from the message storage server a registration notification that notifies registration of a message for the mobile communication terminal in the message storage server, transmits an access request for the registered message via the transmitter-receiver to the message storage server;

wherein the controller, upon receiving via the transmitter-receiver from the message storage server, screen data representing a selection screen that displays message information of at least one message that can be downloaded by the mobile communication terminal, with the message information indicating attributes of the registered message, displays the selection screen represented by the screen data on the display, the message information including an image that best represents the emotion indicated by the emotion data assigned to the message; and

wherein the controller, when one of the at least one message is selected from the selection screen, transmits a download request for the selected message to the message storage server.

5. A mobile communication terminal according to claim 4,

wherein in the selection screen, the message information of the at least one message that can be downloaded by the mobile communication terminal is displayed after being sorted according to a type of emotion indicated by the emotion data of the message.

6. A message storage server of a mobile communication system for relaying communication between at least two mobile communication terminals,

the message storage server comprising:

a message receiver that receives from a first mobile communication terminal a message by speech or video for a second mobile communication terminal;

a storage device that stores the message received by the message receiver;

an emotion recognizer that performs emotion recognition on the message and that assigns emotion data indicating the recognized emotion to the message; and

a confirmation request transmitter that transmits, as a confirmation request, emotion data assigned by the emotion recognizer to the first mobile communication terminal that has transmitted the message.

7. A message storage server according to claim 6, further comprising:

a registration notifier that, upon receiving from the first mobile communication terminal an approval response indicating approval to the emotion data, transmits to the second mobile communication terminal a registration notification of the message that corresponds to the approved emotion data and that is stored in the storage device.

8. A message storage server according to claim 7, further comprising:

a first generator that, upon receiving an access request from the second mobile communication terminal as a response to the registration notification, generates screen data representing a selection screen that displays message information indicating attributes of the stored message; and

a screen data transmitter that transmits the screen data generated by the first generator to the second mobile communication terminal,

wherein the first generator generates, as the selection screen, a screen that displays the message information of the message that can be downloaded by the second mobile communication terminal, after sorting the message information according to a type of emotion indicated by the emotion data of the message.

9. A message storage server according to claim 7, further comprising:

an image extractor that, upon receiving an access request from the second mobile communication terminal in response to the registration notification, extracts from the video data of the message an image that best represents emotion indicated by the emotion data corresponding to the message, in a case in which the stored message is a message by video;

a second generator that generates screen data representing a selection screen that displays an image extracted by the image extractor as at least a part of message information indicating attributes of the stored message; and

a screen data transmitter that transmits the screen data generated by the second generator to the second mobile communication terminal.

10. A message storage server according to claim 9,

wherein the second generator generates, as the selection screen, a screen that displays the message information of the message that can be downloaded by the second mobile communication terminal, after sorting the message information according to a type of emotion indicated by the emotion data of the message.