Service server apparatus, service providing method, and service providing program for providing a service other than a telephone call during the telephone call on a telephone

• US 9,110,888 B2
• Filed: 05/24/2012
• Issued: 08/18/2015
• Est. Priority Date: 05/24/2011
• Status: Active Grant

• Current Assignee: NTT Docomo Incorporated (Nippon Telegraph and Telephone Corporation)
• Original Assignee: NTT Docomo Incorporated (Nippon Telegraph and Telephone Corporation)
• Inventors: Nasu, Kazunori, Shibata, Seiji, Akuzawa, Mari, Uchiyama, Ken
• Primary Examiner(s): REINIER, BARBARA DIANE
• Application Number:
  US13/816,705
• Publication Number:
  US 20140180668A1
• Time in Patent Office: 1,181 Days
• Field of Search: 704 2- 9
• US Class Current: 1/1
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/0638 Interactive procedures
• CPC Class Codes: G10L 2015/221 Announcement of recognition results
• CPC Class Codes: H04M 2201/39 using speech synthesis speech synthesis per se G10L13/00
• CPC Class Codes: H04M 2201/40 using speech recognition speech recognition per se G10L15/00
• CPC Class Codes: H04M 2201/60 Medium conversion
• CPC Class Codes: H04M 2203/2061 Language aspects
• CPC Class Codes: H04M 3/42 Systems providing special services or facilities to subscribers specially adapted for wireless communication networks H04W4/00
• CPC Class Codes: H04M 3/42221 Conversation recording systems at the subscriber's set H04M1/656

Abstract

A service server apparatus is provided which can realize a service as if a speaker has a conversation in a face-to-face manner using existing applications. The service server apparatus includes a service activating unit receiving an instruction for performing a task during a communication, a telephone/call control unit recording a call speech of the speaker during the communication between speech communication terminal devices, a speech recognizing enabler, a text translating enabler, and a speech synthesizing enabler performing the task based on the recorded call speech when the instruction is received, and a communication control enabler providing task data obtained through an performance of the task to the speech communication terminal device. The task data is text data obtained by converting the call speech into a text, translated text data obtained by translating the text data into another language, and speech data obtained by converting the translated text data into a speech.

Claims

1. A service server apparatus connected to a plurality of speech communication terminal devices that perform speech communication with each other via a speech and providing a service by a different task that is a work performed by a different application from a task performed by an application relating to the speech communication between the speech communication terminal devices, the service server apparatus comprising:

an instruction receiving unit configured to receive an instruction for performing the different task during the speech communication;

a recording unit configured to record a call speech of a speaker during the speech communication between the plurality of speech communication terminal devices;

a task performing unit configured to perform the different task based on the call speech recorded by the recording unit, when the instruction receiving unit receives the instruction;

a task data providing unit configured to provide, to each of the plurality of speech communication terminal devices performing the speech communication, task data including text data obtained by performing the different task by the task performing unit and speech data obtained by performing the different task by the task performing unit,

wherein the task performing unit converts the call speech into a text to generate text data, translates the text data into another language to generate translated text data, and converts the translated text data into a speech to generate speech data, and

wherein the task data providing unit provides, to each of the plurality of speech communication terminal devices performing the speech communication, the task data including the text data, the translated text data, and the speech data together with the call speech;

a correction instruction receiving unit configured to receive a correction instruction for the task data transmitted to the speech communication terminal devices; and

a learning unit configured to give a weight to data used when the task data is generated based on the correction instruction received by the correction instruction receiving unit.

4. The service server apparatus according to claim 1 or 2, wherein the task data providing unit transmits the text data in the task data to a text communication terminal device associated with the speech communication terminal device and communicable via a text, instead of the speech communication terminal device.

2. A service server apparatus connected to a plurality of speech communication terminal devices that perform speech communication with each other via a speech and providing a service by a different task that is a work performed by a different application from an application relating to the speech communication between the speech communication terminal devices, the service server apparatus comprising:

an instruction receiving unit configured to receive an instruction to perform the different task during the speech communication;

a recording unit configured to record a call speech of a speaker during the speech communication between the plurality of speech communication terminal devices;

a task performing unit configured to perform the different task based on the call speech recorded by the recording unit, when the instruction receiving unit receives the instruction;

a task data providing unit configured to provide, to each of the plurality of speech communication terminal devices performing the speech communication, task data obtained by performing the different task by the task performing unit,

wherein the task performing unit determines an instruction given by a user of the speech communication terminal device based on the call speech, extracts information registered beforehand in accordance with the instruction, and generates the task data including at least either one of a text and an image using the extracted information, and

wherein the task data providing unit provides, to each of the plurality of speech communication terminal devices performing the speech communication, the task data including at least either one of the text and the image together with the call speech;

a correction instruction receiving unit configured to receive a correction instruction for the task data transmitted to the speech communication terminal devices; and

a learning unit configured to give a weight to data used when the task data is generated based on the correction instruction received by the correction instruction receiving unit.

3. A service server apparatus connected to a plurality of speech communication terminal devices that perform speech communication with each other via a speech and providing a service by a different task that is a work performed by a different application from an application relating to the speech communication between the speech communication terminal devices, the service server apparatus comprising:

an instruction receiving unit configured to receive an instruction to perform the different task during the speech communication;

a recording unit configured to record a call speech of a speaker during the speech communication between the plurality of speech communication terminal devices;

a task performing unit configured to perform the different task based on the call speech recorded by the recording unit, when the instruction receiving unit receives the instruction; and

a task data providing unit configured to provide, to each of the plurality of speech communication terminal devices performing the speech communication, task data obtained by performing the different task by the task performing unit,

wherein the task performing unit determines an instruction given by a user of the speech communication terminal device based on the call speech, extracts information registered beforehand in accordance with the instruction, and generates the task data including at least either one of a text and an image using the extracted information,

wherein the task data providing unit provides, to each of the plurality of speech communication terminal devices performing the speech communication, the task data including at least either one of the text and the image the image together with the call speech, and

wherein when an instruction given by the user is determined to adjust a schedule of the user, the task performing unit extracts schedule information on an activity of the user registered beforehand and a date and an hour at which the activity is to be carried out, and extracts, using the extracted schedule information, a common date and hour at which no activity of the users of the plurality of speech communication terminal devices is commonly registered.

5. A service providing method by a service server apparatus connected to a plurality of speech communication terminal devices for performing speech communication with each other via a speech and providing a service by a different task that is a work performed by a different application from an application relating to the speech communication between the speech communication terminal devices, the service providing method comprising:

an instruction receiving step for receiving an instruction for performing the different task during the speech communication;

a recording step for recording a call speech of a speaker during the speech communication between the plurality of speech communication terminal devices;

a task performing step for converting the call speech into a text to generate text data based on the call speech recorded in the recording step, translating the text data into another language to generate translated text data, and converting the translated text data into a speech to generate speech data when the instruction is received through the instruction receiving step;

a task data providing step for providing, to each of the plurality of speech communication terminal devices performing the speech communication, task data including the text data, the translated text data, and the speech data together with the call speech;

a correction instruction receiving step for receiving a correction instruction for the task data transmitted to the speech communication terminal devices; and

a learning step for giving a weight to data used when the task data is generated based on the correction instruction received in the correction instruction receiving step.

6. A service providing method by a service server apparatus connected to a plurality of speech communication terminal devices for performing speech communication with each other via a speech and providing a service by a different task that is a work performed by a different application from an application relating to the speech communication between the speech communication terminal devices, the service providing method comprising:

an instruction receiving step for receiving an instruction for performing the different task during the speech communication;

a recording step for recording a call speech of a speaker during the speech communication between the plurality of speech communication terminal devices;

a task performing step for determining an instruction given by a user of the speech communication terminal device based on the call speech recorded through the recording step, extracting information registered beforehand based on the instruction, and generating task data including at least either one of a text and an image using the extracted information when the instruction is received through the instruction receiving step;

a task data providing step for providing, to each of the plurality of speech communication terminal devices performing the speech communication, the task data including at least either one of the text and the image together with the call speech;

a correction instruction receiving step for receiving a correction instruction for the task data transmitted to the speech communication terminal devices; and

a learning step for giving a weight to data used when the task data is generated based on the correction instruction received in the correction instruction receiving step.

7. A non-transitory computer readable medium storing a service providing program performed by a service server apparatus connected to a plurality of speech communication terminal devices for performing speech communication with each other via a speech and providing a service by a different task that is a work performed by a different application from an application relating to the speech communication between the speech communication terminal devices, the service providing program allowing a computer to realize:

an instruction receiving function of receiving an instruction for performing the different task during the speech communication;

a recording function of recording a call speech of a speaker during the speech communication between the plurality of speech communication terminal devices;

a task performing function of converting the call speech into a text to generate text data based on the call speech recorded by the recording function, translating the text data into another language to generate translated text data, and converting the translated text data into a speech to generate speech data when the instruction is received by the instruction receiving function;

a task data providing function of providing, to each of the plurality of speech communication terminal devices performing the speech communication, task data including the text data, the translated text data, and the speech data together with the call speech;

a correction instruction receiving function of receiving a correction instruction for the task data transmitted to the speech communication terminal devices; and

a learning function of giving a weight to data used when the task data is generated based on the correction instruction received by the correction instruction receiving function.

8. A non-transitory computer readable medium storing a service providing program performed by a service server apparatus connected to a plurality of speech communication terminal devices for performing speech communication with each other via a speech and providing a service by a different task that is a work performed by a different application from an application relating to the speech communication between the speech communication terminal devices, the service providing program allowing a computer to realize:

an instruction receiving function of receiving an instruction for performing the different task during the speech communication;

a recording function of recording a call speech of a speaker during the speech communication between the plurality of speech communication terminal devices;

a task performing function of determining an instruction given by a user of the speech communication terminal device based on the call speech recorded by the recording function, extracting information registered beforehand based on the instruction, and generating task data including at least either one of a text and an image using the extracted information when the instruction is received by the instruction receiving function;

a task data providing function of providing, to each of the plurality of speech communication terminal devices performing the speech communication, the task data including at least either one of the text and the image together with the call speech

a correction instruction receiving function of receiving a correction instruction for the task data transmitted to the speech communication terminal devices; and

a learning function of giving a weight to data used when the task data is generated based on the correction instruction received by the correction instruction receiving function.

9. A service providing method by a service server apparatus connected to a plurality of speech communication terminal devices for performing speech communication with each other via a speech and providing a service by a different task that is a work performed by a different application from an application relating to the speech communication between the speech communication terminal devices, the service providing method comprising:

an instruction receiving step for receiving an instruction for performing the different task during the speech communication;

a recording step for recording a call speech of a speaker during the speech communication between the plurality of speech communication terminal devices;

a task performing step for determining an instruction given by a user of the speech communication terminal device based on the call speech recorded through the recording step, extracting information registered beforehand based on the instruction, and generating task data including at least either one of a text and an image using the extracted information when the instruction is received through the instruction receiving step; and

a task data providing step for providing, to each of the plurality of speech communication terminal devices performing the speech communication, the task data including at least either one of the text and the image together with the call speech;

wherein when an instruction given by the user is determined to adjust a schedule of the user, further comprising an extracting step of extracting schedule information on an activity of the user registered beforehand and a date and an hour at which the activity is to be carried out, and extracting, using the extracted schedule information, a common date and hour at which no activity of the users of the plurality of speech communication terminal devices is commonly registered.

10. A non-transitory computer readable medium storing a service providing program performed by a service server apparatus connected to a plurality of speech communication terminal devices for performing speech communication with each other via a speech and providing a service by a different task that is a work performed by a different application from an application relating to the speech communication between the speech communication terminal devices, the service providing program allowing a computer to realize:

an instruction receiving function of receiving an instruction for performing the different task during the speech communication;

a recording function of recording a call speech of a speaker during the speech communication between the plurality of speech communication terminal devices;

a task performing function of determining an instruction given by a user of the speech communication terminal device based on the call speech recorded by the recording function, extracting information registered beforehand based on the instruction, and generating task data including at least either one of a text and an image using the extracted information when the instruction is received by the instruction receiving function; and

a task data providing function of providing, to each of the plurality of speech communication terminal devices performing the speech communication, the task data including at least either one of the text and the image together with the call speech;

wherein when an instruction given by the user is determined to adjust a schedule of the user, an extraction function of extracting schedule information on an activity of the user registered beforehand and a date and an hour at which the activity is to be carried out, and extracting, using the extracted schedule information, a common date and hour at which no activity of the users of the plurality of speech communication terminal devices is commonly registered.