METHOD AND SYSTEM FOR OPERATING COMMUNICATION SERVICE

• US 20130304457A1
• Filed: 05/07/2013
• Published: 11/14/2013
• Est. Priority Date: 05/08/2012
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: KANG, Sangki, KO, Jungwan, KONG, Kichoon, KIM, Kyungtae, LEE, Sanghoon
• Granted Patent: US 9,344,878 B2
• Time in Patent Office: Days
• US Class Current: 704/201
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04M 1/72433 for voice messaging, e.g. dictaphones for answering incoming calls H04M1/64
• CPC Class Codes: H04M 1/72436 for text messaging, e.g. short messaging services [SMS] or e-mails
• CPC Class Codes: H04M 1/72439 for image or video messaging
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions
• CPC Class Codes: H04M 1/72457 according to geographic location
• CPC Class Codes: H04M 2201/39 using speech synthesis
• CPC Class Codes: H04M 2201/40 using speech recognition
• CPC Class Codes: H04M 2201/60 Medium conversion
• CPC Class Codes: H04M 2203/253 where a visual mode is used instead of a voice mode
• CPC Class Codes: H04M 3/42382 Text-based messaging services in telephone networks such as PSTN/ISDN, e.g. User-to-User Signalling or Short Message Service for fixed networks
• CPC Class Codes: H04W 8/22 Processing or transfer of terminal data, e.g. status or physical capabilities

Abstract

An operation method capable of adaptively operating at least one of a Speech To Text (STT) service and a Text To Speech (TTS) service according to setting or user operation and a system thereof are provided. The method includes requesting a specific type of a communication service connection to a reception side terminal by a transmission side terminal, and performing an operation of at least one of a speech to text service providing speech recognition based text and a text to speech service converting the text into speech data between the reception side terminal and the transmission side terminal, and includes one of recognizing speech data provided from the transmission side terminal and converting the speech data into a text based on a first speech process supporting device connected to the transmission side terminal.

Claims

1. A system for operating a communication service, the system comprising:

a transmission side terminal for requesting connection of the communication service, and for converting reception data or user input data according to at least one of preset input/output modes and input/output modes determined by a user after the communication service is connected;

a reception side terminal for receiving a communication service connection acceptance request according to the request the connection of the communication service, for starting the communication service using the at least one of the preset input/output modes and the input/output modes determined by the user when the communication service connection acceptance request is accepted, for converting the reception data and the user input data or receiving converted data by transmitting the user input data to a speech process supporting device for converting the user input data according a type of the reception data and the input/output modes, for displaying conversion data corresponding to the reception data, and for transmitting conversion data corresponding to the user input data; and

a speech process supporting device for converting input speech data into text data and for outputting the converted text data or converting input text data into speech data to output the converted speech data.

2. The system of claim 1, wherein the transmission side terminal transmits a message requesting to operate at least one of a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data to the reception side terminal according to the input/output modes.

3. The system of claim 1, wherein the reception side terminal requests to the transmission side terminal to operate at least one of a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data to the reception side terminal when the communication service connection request is received from the transmission side terminal according to the input/output modes.

4. The system of claim 1, wherein at least one of the transmission side terminal and the reception side terminal requests to automatically operate at least one of a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data based on at least one of network evaluation information including at least one of a loss rate of a data packet and a transmission delay degree transceived between the transmission side terminal and the reception side terminal, peripheral environment information of the transmission side terminal, and peripheral environment information of the reception side terminal.

5. The system of claim 4, wherein the peripheral environment information comprises at least one of peripheral noise information, moving information, location information, illumination information, and current time information of at least one of the transmission side terminal and the reception side terminal.

6. The system of claim 1, wherein the transmission side terminal and the reception side terminal form a separate channel for supporting at least one of a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data in a state in which a chatting service channel is formed.

7. The system of claim 1, wherein the transmission side terminal and the reception side terminal form a separate channel for supporting at least one of a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data in a state in which an image call service channel is formed.

8. The system of claim 7, wherein the transmission side terminal combines image data to be transmitted for supporting the image call service with text generated based on an operation of the STT service, and transmits the combined data to the reception side terminal.

9. The system of claim 8, wherein the transmission side terminal synchronizes the text with the image data using time stamp information collected when a speech signal corresponding to the generated text is collected and time stamp information of the image data collected when the speech signal is collected.

10. The system of claim 8, wherein the transmission side terminal controls such that a previous speech recognized text is synchronized with the image data for a predetermined time before text obtained by recognizing a newly input speech signal, and controls such that an input text when the speech recognized text is input is synchronized with the image data.

11. The system of claim 1, wherein the transmission side terminal comprises a first memory including at least one of a speech recognition database recognizing a collected speech signal and a speech conversion database converting an input text into a speech.

12. The system of claim 11, wherein the transmission side terminal controls such that at least one of a Speech To Text (STT) service providing a speech based on the speech recognition database as text and a Text To Speech (TTS) service converting text into speech data is operated according to at least one of a connection state and a user operation of the speech process supporting device.

13. The system of claim 1, wherein the transmission side terminal comprises a first display unit including at least one of:

a pop-up window for selecting at least one from a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data;

a picture for guiding connection of the speech process supporting device according to the operation of the STT service; and

a picture including at least one of a region for outputting a speech recognized text and a region text created and transmitted by the reception side terminal.

14. The system of claim 1, wherein the reception side terminal comprises a second display unit including at least one of:

a pop-up window for selecting at least one from a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data when the call connection request is received; and

a picture including at least one of a region for outputting text provided from the transmission side terminal, a region for outputting text input through a text creator, and the text creator for creating the text.

15. A method of operating a communication service, the method comprising:

receiving a communication service connection request;

forming a communication path by accepting the communication service connection request;

setting input/output modes according to input/output modes determined by a user when the input/output modes are previously set or when the communication service connection request is accepted;

receiving data from another party;

converting the received data or receiving and displaying converted data by transmitting the received data to a speech process supporting device for converting the received data according to a type of the received data and the input/output modes;

receiving user input data; and

converting the received data or receiving converted data by transmitting the received data to the speech process supporting device for converting the received data according to the type of the received data and the input/output modes, and transmitting the converted data to the other party.

16. The method of claim 15, wherein the receiving of the communication service connection request comprises transmitting a message requesting to operate at least one of a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data to the reception side terminal by the transmission side terminal according to the input/output modes.

17. The method of claim 15, wherein the receiving of the communication service connection request comprises requesting to the transmission side terminal to operate at least one of a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data to the reception side terminal by the reception side terminal when the communication service connection request is received from the transmission side terminal according to the input/output modes.

18. The method of claim 15, wherein the receiving of the communication service connection request comprises requesting to automatically operate at least one of a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data based on at least one of network evaluation information including at least one of a loss rate of a data packet and a transmission delay degree transceived between the transmission side terminal and the reception side terminal, peripheral environment information of the transmission side terminal, and peripheral environment information of the reception side terminal by at least one of the transmission side terminal and the reception side terminal.

19. The method of claim 18, wherein the peripheral environment information comprises at least one of peripheral noise information, moving information, location information, illumination information, and current time information of at least one of the transmission side terminal and the reception side terminal.

20. The method of claim 15, wherein the transmission side terminal and the reception side terminal form a separate channel for supporting at least one of a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data in a state in which a chatting service channel is formed.

21. The method of claim 15, wherein the forming of the communication path comprises forming a separate channel for supporting at least one of a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data in a state in which an image call service channel is formed.

22. The method of claim 21, wherein the transmitting of the converted data comprises combining image data to be transmitted for supporting the image call service with text generated based on an operation of the STT service to transmit the combined data to the reception side terminal by the transmission side terminal.

23. The method of claim 21, further comprising synchronizing the text with the image data using time stamp information collected when a speech signal corresponding to the generated text is collected and time stamp information of the image data collected when the speech signal is collected by the transmission side terminal.

24. The method of claim 21, further comprising controlling such that a previous speech recognized text is synchronized with the image data for a predetermined time before text obtained by recognizing a newly input speech signal, and controlling such that an input text when the speech recognized text is input is synchronized with the image data by the transmission side terminal.

25. The method of claim 15, wherein the transmitting of the converted data comprises providing at least one of a Speech To Text (STT) service providing a speech based on the speech recognition database as text and a Text To Speech (TTS) service converting text into speech data is operated according to at least one of a connection state and a user operation of the speech process supporting device by the transmission side terminal.

26. The method of claim 15, further comprising displaying at least one of:

a pop-up window selecting at least one from a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data;

a picture guiding connection of the speech process supporting device according to the operation of the STT service; and

a picture including at least one of a region outputting a speech recognized text and a region text created and transmitted from the reception side terminal by the transmission side terminal.

27. The method of claim 25, further comprising outputting at least one of:

a pop-up window selecting at least one from a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data when the call connection request is received; and

a picture include at least one of a region outputting text provided from the transmission side terminal, a region outputting text input through a text creator, and the text creator for creating the text by the reception side terminal.

28. A system for operating a communication service, the system comprising:

a transmission side terminal for requesting connection of the communication service, for receiving information about input/output modes of another party, for converting the user input data and data received from the other party or receiving converted data by transmitting the data to a speech process supporting device for converting the user input data according to the received input/output modes of the other party after the communication service is connected, for outputting conversion data corresponding to the received data, and for transmitting conversion data corresponding to the user input data;

a reception side terminal for receiving a communication service connection acceptance request according to the request the connection of the communication service, for starting the communication service using at least one of preset input/output modes or input/output modes determined by the user when the communication service connection acceptance request is accepted, for transmitting information about the input/output modes to a transmission side, for displaying the conversion data received from the transmission side, and for transmitting the user input data; and

a speech process supporting device for providing at least one of a Speech To Text (STT) service providing input speech data as text data and a Text To Speech (TTS) service converting text into speech data.

29. The system of claim 28, wherein the reception side terminal requests to automatically operate at least one of a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data based on at least one of network evaluation information including at least one of a loss rate of a data packet and a transmission delay degree transceived between the transmission side terminal and the reception side terminal, peripheral environment information of the transmission side terminal, and peripheral environment information of the reception side terminal.

30. The system of claim 29, wherein the peripheral environment information comprises at least one of peripheral noise information, moving information, location information, illumination information, and current time information of at least one of the transmission side terminal and the reception side terminal.

31. The system of claim 28, wherein the reception side terminal forms at least one of a separate channel for supporting at least one of an STT service and a TTS service in a state in which a channel service channel is formed with the transmission side terminal, and a separate channel for supporting at least one of the STT service and the TTS service in a state in which an image call service channel is formed.

32. The system of claim 31, wherein the reception side terminal extracts time stamp information collected when a speech signal corresponding to generated text is collected and time stamp information of the image data collected for the image call service when the speech signal is collected from data provided from the transmission side terminal to synchronize the text with the image data.

33. The system of claim 28, wherein the reception side terminal comprises a second memory including at least one of a speech recognition database recognizing a collected speech signal and a speech conversion database converting an input text into a speech.

34. The system of claim 33, wherein the reception side terminal controls such that at least one of an STT service and a TTS is operated according to at least one of a connection state and a user operation of the speech process supporting device.

35. The system of claim 28, wherein the reception side terminal comprises a second display unit including at least one of:

a pop-up window selecting at least one from a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data when the call connection request is received; and

a picture include at least one of a region outputting text provided from the transmission side terminal, a region outputting text input through a text creator, and the text creator for creating the text.

36. A method of operating a communication service, the method comprising:

transmitting a communication service connection request;

setting input/output modes according to input/output modes determined by a user when the input/output modes are previously set or when the communication service connection request is accepted;

receiving acceptance with respect to the communication service connection request and information about the input/output modes of another party;

receiving user input data;

converting the received user input data or receiving and transmitting converted data by transmitting the received data to a speech process supporting device for converting the received user input data according to a type of the received user input data and the input/output modes;

receiving data from another party; and

converting the received data or receiving and displaying converted data by transmitting the data to the speech process supporting device for converting the received data according to the type of the received data and the input/output modes determined by the user.

37. The method of claim 36, wherein the setting of the input/output modes comprises:

collecting at least one of network evaluation information including at least one of a loss rate of a data packet and a transmission delay degree transceived between a transmission side terminal performing the communication service connection request and a reception side terminal receiving the communication service connection request, peripheral environment information of the transmission side terminal, and peripheral environment information of the reception side terminal; and

requesting to operate at least one of a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data based on the information by the reception side terminal.

38. The system of claim 37, wherein the peripheral environment information comprises at least one of peripheral noise information, moving information, location information, illumination information, and current time information of at least one of the transmission side terminal and the reception side terminal.

39. The method of claim 36, further comprising at least one of:

forming a separate channel for supporting at least one of a Speech To Text (STT) service providing a speech recognition based text and a Text To Speech (TTS) service converting text into speech data in a state in which a chatting service channel is formed with the transmission side terminal by the reception side terminal receiving the communication service connection request; and

forming a separate channel for supporting at least one of a STT service providing a speech recognition based text and a TTS service converting text into speech data in a state in which an image call service channel is formed by the reception side terminal.

40. The method of claim 39, further comprising:

extracting time stamp information collected when a speech signal corresponding to the generated text is collected and time stamp information of the image data collected for the image call service when the speech signal is collected from data provided from the transmission side terminal by the reception side terminal; and

synchronizing the text with the image data by the reception side terminal.

41. The method of claim 36, further comprising performing at least one of a Speech To Text (STT) service converting speech data corresponding to a speech signal into text data or a Text To Speech (TTS) service converting the text data into the speech data using a speech recognition database and a speech conversion database stored in a second memory by the reception side terminal receiving the communication service connection request.

42. The method of claim 41, further comprising controlling such that at least one of the STT service based on the speech recognition database and the TTS service based on the speech conversion database is operated according to at least one of a connection state and a user operation of the speech process supporting device.

43. The method of claim 39, further comprising at least one of:

displaying a pop-up window selecting at least one of a Speech To Text (STT) service converting speech data into text data and a Text To Speech (TTS) service converting the text data into the speech data when the call connection request is received;

displaying a picture guiding connection of the speech process supporting device; and

displaying a picture including at least one of a region outputting the input text and a region outputting text generated according to the operation of the STT service.