Method and system for operating communication service

• US 9,344,878 B2
• Filed: 05/07/2013
• Issued: 05/17/2016
• Est. Priority Date: 05/08/2012
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kang, Sangki, Ko, Jungwan, Kong, Kichoon, Kim, Kyungtae, Lee, Sanghoon
• Primary Examiner(s): BRYAR, JEREMIAH A
• Application Number:
  US13/888,726
• Publication Number:
  US 20130304457A1
• Time in Patent Office: 1,106 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04M 1/72433 for voice messaging, e.g. dictaphones for answering incoming calls H04M1/64
• CPC Class Codes: H04M 1/72436 for text messaging, e.g. short messaging services [SMS] or e-mails
• CPC Class Codes: H04M 1/72439 for image or video messaging
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions
• CPC Class Codes: H04M 1/72457 according to geographic location
• CPC Class Codes: H04M 2201/39 using speech synthesis speech synthesis per se G10L13/00
• CPC Class Codes: H04M 2201/40 using speech recognition speech recognition per se G10L15/00
• CPC Class Codes: H04M 2201/60 Medium conversion
• CPC Class Codes: H04M 2203/253 where a visual mode is used instead of a voice mode
• CPC Class Codes: H04M 3/42382 Text-based messaging services in telephone networks such as PSTN/ISDN, e.g. User-to-User Signalling or Short Message Service for fixed networks
• CPC Class Codes: H04W 8/22 Processing or transfer of terminal data, e.g. status or physical capabilities

Abstract

An operation method capable of adaptively operating at least one of a Speech To Text (STT) service and a Text To Speech (TTS) service according to setting or user operation and a system thereof are provided. The method includes requesting a specific type of a communication service connection to a reception side terminal by a transmission side terminal, and performing an operation of at least one of a speech to text service providing speech recognition based text and a text to speech service converting the text into speech data between the reception side terminal and the transmission side terminal, and includes one of recognizing speech data provided from the transmission side terminal and converting the speech data into a text based on a first speech process supporting device connected to the transmission side terminal.

Claims

1. A system for operating a communication service, the system comprising:

a transmission side terminal for requesting connection of the communication service, and for converting reception data or user input data according to at least one of input/output modes and input/output modes determined by a user after the communication service is connected;

a reception side terminal for receiving a communication service connection acceptance request according to the request the connection of the communication service, for starting the communication service using the at least one of input/output modes and input/output modes determined by a user when the communication service connection acceptance request is accepted, for transmitting reception data or user input data to a speech process supporting device to convert the user input data or the reception data, for displaying conversion data corresponding to the reception data, and for transmitting conversion data corresponding to the user input data to the transmission side terminal; and

a speech process supporting device for receiving the reception data or the user input data from the reception side terminal, for converting input speech data corresponding the reception data or the user input data into text data or converting input text data corresponding the reception data or the user input data into speech data to create conversion data corresponding to the reception data or the user input data, and for transmitting the conversion data to the reception side terminal.

2. The system of claim 1, wherein the transmission side terminal transmits a message requesting to operate at least one of a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data to the reception side terminal according to the input/output modes.

3. The system of claim 1, wherein the reception side terminal requests to the transmission side terminal to operate at least one of a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data to the reception side terminal when the communication service connection request is received from the transmission side terminal according to the input/output modes.

4. The system of claim 1, wherein at least one of the transmission side terminal and the reception side terminal requests to automatically operate at least one of a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data based on at least one of network evaluation information including at least one of a loss rate of a data packet and a transmission delay degree transceived between the transmission side terminal and the reception side terminal, peripheral environment information of the transmission side terminal, and peripheral environment information of the reception side terminal.

5. The system of claim 4, wherein the peripheral environment information comprises at least one of peripheral noise information, moving information, location information, illumination information, and current time information of at least one of the transmission side terminal and the reception side terminal.

6. The system of claim 1, wherein the transmission side terminal and the reception side terminal form a separate channel for supporting at least one of a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data in a state in which a chatting service channel is formed.

7. The system of claim 1, wherein the transmission side terminal and the reception side terminal form a separate channel for supporting at least one of a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data in a state in which an image call service channel is formed.

8. The system of claim 7, wherein the transmission side terminal combines image data to be transmitted for supporting the image call service with text generated based on an operation of the STT service, and transmits the combined data to the reception side terminal.

9. The system of claim 8, wherein the transmission side terminal synchronizes the text with the image data using time stamp information collected when a speech signal corresponding to the generated text is collected and time stamp information of the image data collected when the speech signal is collected.

10. The system of claim 8, wherein the transmission side terminal controls such that a previous speech recognized text is synchronized with the image data for a predetermined time before text obtained by recognizing a newly input speech signal, and controls such that an input text when the speech recognized text is input is synchronized with the image data.

11. The system of claim 1, wherein the transmission side terminal comprises a first memory including at least one of a speech recognition database recognizing a collected speech signal and a speech conversion database converting an input text into a speech.

12. The system of claim 11, wherein the transmission side terminal controls such that at least one of a Speech To Text (STT) service providing a speech based on the speech recognition database as text and a Text To Speech (TTS) service converting text into speech data is operated according to at least one of a connection state and a user operation of the speech process supporting device.

13. The system of claim 1, wherein the transmission side terminal comprises a first display unit including at least one of:

a pop-up window for selecting at least one from a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data;

a picture for guiding connection of the speech process supporting device according to the operation of the STT service; and

a picture including at least one of a region for outputting a speech recognized text and a region text created and transmitted by the reception side terminal.

14. The system of claim 1, wherein the reception side terminal comprises a second display unit including at least one of:

a pop-up window for selecting at least one from a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data when the call connection request is received; and

a picture including at least one of a region for outputting text provided from the transmission side terminal, a region for outputting text input through a text creator, and the text creator for creating the text.

15. A method of operating a communication service, the method comprising:

receiving, by a reception side terminal, a communication service connection request;

forming, by the reception side terminal, a communication path with a transmission side terminal by accepting the communication service connection request;

identifying, by the reception side terminal, input/output modes;

receiving, by the reception side terminal, reception data from the transmission side terminal or user input data from a user;

transmitting, by the reception side terminal, the reception data or the user input data to a speech process supporting device to convert the reception data or the user input;

receiving, by the reception side terminal, conversion data corresponding to the reception data or the user input data from the speech process supporting device, the conversion data being created by converting input speech data corresponding the reception data or the user input data into text data or converting input text data corresponding the reception data or the user input data into speech data;

displaying, by the reception side terminal, conversion data corresponding to the reception data; and

transmitting, by the reception side terminal, conversion data corresponding to the user input data to the transmission side terminal.

16. The method of claim 15, wherein the receiving of the communication service connection request comprises transmitting a message requesting to operate at least one of a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data to the reception side terminal by the transmission side terminal according to the input/output modes.

17. The method of claim 15, wherein the receiving of the communication service connection request comprises requesting to the transmission side terminal to operate at least one of a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data to the reception side terminal by the reception side terminal when the communication service connection request is received from the transmission side terminal according to the input/output modes.

18. The method of claim 15, wherein the receiving of the communication service connection request comprises requesting to automatically operate at least one of a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data based on at least one of network evaluation information including at least one of a loss rate of a data packet and a transmission delay degree transceived between the transmission side terminal and the reception side terminal, peripheral environment information of the transmission side terminal, and peripheral environment information of the reception side terminal by at least one of the transmission side terminal and the reception side terminal.

19. The method of claim 18, wherein the peripheral environment information comprises at least one of peripheral noise information, moving information, location information, illumination information, and current time information of at least one of the transmission side terminal and the reception side terminal.

20. The method of claim 15, wherein the transmission side terminal and the reception side terminal form a separate channel for supporting at least one of a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data in a state in which a chatting service channel is formed.

21. The method of claim 15, wherein the forming of the communication path comprises forming a separate channel for supporting at least one of a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data in a state in which an image call service channel is formed.

22. The method of claim 21, wherein the transmitting of the conversion data comprises combining image data to be transmitted for supporting the image call service with text generated based on an operation of the STT service to transmit the combined data to the reception side terminal by the transmission side terminal.

23. The method of claim 21, further comprising synchronizing the text with the image data using time stamp information collected when a speech signal corresponding to the generated text is collected and time stamp information of the image data collected when the speech signal is collected by the transmission side terminal.

24. The method of claim 21, further comprising controlling such that a previous speech recognized text is synchronized with the image data for a predetermined time before text obtained by recognizing a newly input speech signal, and controlling such that an input text when the speech recognized text is input is synchronized with the image data by the transmission side terminal.

25. The method of claim 15, wherein the transmitting of the conversion data comprises providing at least one of a Speech To Text (STT) service providing a speech based on the speech recognition database as text and a Text To Speech (TTS) service converting text into speech data is operated according to at least one of a connection state and a user operation of the speech process supporting device by the transmission side terminal.

26. The method of claim 25, further comprising outputting at least one of:

a pop-up window selecting at least one from a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data when the call connection request is received; and

a picture include at least one of a region outputting text provided from the transmission side terminal, a region outputting text input through a text creator, and the text creator for creating the text by the reception side terminal.

27. The method of claim 15, further comprising displaying at least one of:

a pop-up window selecting at least one from a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data;

a picture guiding connection of the speech process supporting device according to the operation of the STT service; and

a picture including at least one of a region outputting a speech recognized text and a region text created and transmitted from the reception side terminal by the transmission side terminal.

28. A system for operating a communication service, the system comprising:

a transmission side terminal for requesting connection of the communication service, for receiving information about input/output modes of a reception side terminal, for transmitting user input data or reception data to a speech process supporting device to convert the user input data or the reception data after the communication service is connected, for outputting conversion data corresponding to the reception data, and for transmitting conversion data corresponding to the user input data to a reception side terminal;

a reception side terminal for receiving a communication service connection acceptance request according to the request the connection of the communication service, for starting the communication service using at least one of input/output modes or input/output modes determined by a user when the communication service connection acceptance request is accepted, for transmitting information about the input/output modes to a transmission side terminal, for displaying the conversion data received from the transmission side terminal, and for transmitting user input data to the transmission side terminal; and

a speech process supporting device for receiving the reception data or the user input data from the transmission side terminal, for converting input speech data corresponding the reception data or the user input data into text data or converting input text data corresponding the reception data or the user input data into speech data to create conversion data corresponding to the reception data or the user input data, and for transmitting the conversion data to the reception side terminal.

29. The system of claim 28, wherein the reception side terminal requests to automatically operate at least one of a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data based on at least one of network evaluation information including at least one of a loss rate of a data packet and a transmission delay degree transceived between the transmission side terminal and the reception side terminal, peripheral environment information of the transmission side terminal, and peripheral environment information of the reception side terminal.

30. The system of claim 29, wherein the peripheral environment information comprises at least one of peripheral noise information, moving information, location information, illumination information, and current time information of at least one of the transmission side terminal and the reception side terminal.

31. The system of claim 28, wherein the reception side terminal forms at least one of a separate channel for supporting at least one of an STT service and a TTS service in a state in which a channel service channel is formed with the transmission side terminal, and a separate channel for supporting at least one of the STT service and the TTS service in a state in which an image call service channel is formed.

32. The system of claim 31, wherein the reception side terminal extracts time stamp information collected when a speech signal corresponding to generated text is collected and time stamp information of the image data collected for the image call service when the speech signal is collected from data provided from the transmission side terminal to synchronize the text with the image data.

33. The system of claim 28, wherein the reception side terminal comprises a second memory including at least one of a speech recognition database recognizing a collected speech signal and a speech conversion database converting an input text into a speech.

34. The system of claim 33, wherein the reception side terminal controls such that at least one of an STT service and a TTS is operated according to at least one of a connection state and a user operation of the speech process supporting device.

35. The system of claim 28, wherein the reception side terminal comprises a second display unit including at least one of:

a pop-up window selecting at least one from a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data when the call connection request is received; and

a picture include at least one of a region outputting text provided from the transmission side terminal, a region outputting text input through a text creator, and the text creator for creating the text.

36. A method of operating a communication service, the method comprising:

transmitting, by a transmission side terminal, a communication service connection request to a reception side terminal;

identifying, by a transmission side terminal, input/output modes;

receiving, by a transmission side terminal, acceptance with respect to the communication service connection request and information about the input/output modes of reception side terminal from the reception side terminal;

receiving, by a transmission side terminal, user input data from a user or reception data from the transmission side terminal;

transmitting, by a transmission side terminal, the reception data or the user input data to a speech process supporting device to convert the received user input data or the reception data;

receiving, by a transmission side terminal, conversion data corresponding to the reception data or the user input data from the speech process supporting device, the conversion data being created by converting input speech data corresponding the reception data or the user input data into text data or converting input text data corresponding the reception data or the user input data into speech data;

displaying, by a transmission side terminal, conversion data corresponding to the reception data; and

transmitting, by a transmission side terminal, conversion data corresponding to the user input data to the reception side terminal.

37. The method of claim 36, wherein the identifying of the input/output modes comprises:

collecting at least one of network evaluation information including at least one of a loss rate of a data packet and a transmission delay degree transceived between the transmission side terminal performing the communication service connection request and the reception side terminal receiving the communication service connection request, peripheral environment information of the transmission side terminal, and peripheral environment information of the reception side terminal; and

requesting to operate at least one of a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data based on the information by the reception side terminal.

38. The system of claim 37, wherein the peripheral environment information comprises at least one of peripheral noise information, moving information, location information, illumination information, and current time information of at least one of the transmission side terminal and the reception side terminal.

39. The method of claim 36, further comprising at least one of:

forming a separate channel for supporting at least one of a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data in a state in which a chatting service channel is formed with the transmission side terminal by the reception side terminal receiving the communication service connection request; and

forming a separate channel for supporting at least one of a STT service providing a speech recognition based text and a TTS service converting text into speech data in a state in which an image call service channel is formed by the reception side terminal.

40. The method of claim 39, further comprising:

extracting time stamp information collected when a speech signal corresponding to the generated text is collected and time stamp information of the image data collected for the image call service when the speech signal is collected from data provided from the transmission side terminal by the reception side terminal; and

synchronizing the text with the image data by the reception side terminal.

41. The method of claim 39, further comprising at least one of:

displaying a pop-up window selecting at least one of a Speech To Text (STT) service converting speech data into text data and a Text To Speech (TTS) service converting the text data into the speech data when the call connection request is received;

displaying a picture guiding connection of the speech process supporting device; and

displaying a picture including at least one of a region outputting the input text and a region outputting text generated according to the operation of the STT service.

42. The method of claim 36, further comprising performing at least one of a Speech To Text (STT) service converting speech data corresponding to a speech signal into text data or a Text To Speech (TTS) service converting the text data into the speech data using a speech recognition database and a speech conversion database stored in a second memory by the reception side terminal receiving the communication service connection request.

43. The method of claim 42, further comprising controlling such that at least one of the STT service based on the speech recognition database and the TTS service based on the speech conversion database is operated according to at least one of a connection state and a user operation of the speech process supporting device.