Smartphone-based telephone translation system

• US 10,776,588 B2
• Filed: 07/23/2018
• Issued: 09/15/2020
• Est. Priority Date: 05/09/2018
• Status: Active Grant

• Current Assignee: Shenzhen Zhiyuan Technology Co., Ltd.
• Original Assignee: Shenzhen Zhiyuan Technology Co., Ltd.
• Inventors: Yu, Liang, Wang, Chuan
• Primary Examiner(s): Thomas-Homescu, Anne L
• Application Number:
  US16/043,104
• Publication Number:
  US 20190347331A1
• Time in Patent Office: 785 Days
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 13/027 Concept to speech synthesisers; Generation of natural phrases from machine-based concepts generation of parameters for speech synthesis out of text G10L13/08
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04M 1/6033 for providing handsfree use or a loudspeaker mode in telephone sets arrangements for preventing acoustic echo H04M9/08
• CPC Class Codes: H04M 1/72478 for hearing-impaired users
• CPC Class Codes: H04M 1/72484 wherein functions are triggered by incoming communication events
• CPC Class Codes: H04M 2250/58 including a multilanguage function
• CPC Class Codes: H04M 2250/74 with voice recognition means voice activated dialling H04M1/271; voice recognition algorithms G10L15/00

Abstract

A smartphone-based telephone translation system includes a translation machine that starts a call translation process when the call translation is set to ON. The system detects an incoming call or dials, translates a voice signal to obtain a translation text for the ON call state. The voice signal of the caller is received and recognized. A voice signal is translated to obtain a translation text for the ON call state. The localized voice signal is recognized, using a TTS machine to obtain a reply voice signal and sent to the caller. The system performs voice recognition and text translation on the call voice, translates the language into localized language, and performs the function of calling between different languages in real time, so as to realize a telephone communication between two persons speaking different languages. The system supports communication with a hearing-impaired person or deaf-mute person.

Claims

1. A smartphone-based telephone translation system, comprising a first translation machine and a second translation machine, wherein:

the first translation machine comprises a first processor, a first input device, a first output device and a first memory that are connected with one another;

the first memory is used to store a computer program which includes first program instructions;

the first processor is configured to call the first program instructions for performing a step of starting a first call translation process for an ON call translation state;

the first translation machine detects an incoming call or dials;

the first call translation process includes;

translating a voice signal to obtain a translation text for the ON call translation state and the voice signal of the caller is received and recognized, and displayed;

displaying, based on the translation text, using a Text-to-Speech (TTS) machine for a pronunciation to obtain a reply voice signal; and

sending the reply voice signal to the caller;

the second translation machine comprises a second processor, a second input device, a second output device, a second memory and a server that are connected with one another;

the second memory is used to store a computer program which includes second program instructions;

the second processor is configured to;

call the second program instructions for performing a step of starting a second call translation process for an ON call translation state;

after receiving and recognizing a voice signal, call the second program instructions for performing a step of sending the received and recognized voice signal to the server; and

receive a translation text returned by the server;

the server is configured to receive a speech signal from a third translation machine, translate the voice signal to obtain a translation text, and send the translation text to the third translation machine;

the second translation machine detects an incoming call or dials;

the second call translation process includes;

translating a voice signal to obtain a translation text for the ON call translation state and the voice signal of the caller is received and recognized, and displayed;

displaying, based on the translation text, using a Text-to-Speech (TTS) machine for a pronunciation to obtain a reply voice signal; and

sending the reply voice signal to the caller;

the first translation machine adopts offline translation, and performs translation by vocabulary built in the first translation machine;

the second translation machine provides an online translation function, wherein a voice signal is uploaded to the server and the server performs online translation on a network;

the first translation machine implements translation between Chinese and English;

the second translation machine implements translation between Chinese and English;

in response to the first translation machine'"'"'s reply to a message, the first translation machine'"'"'s local identification thread starts to translate the message and then pronounces through the TTS for sending to the second translation machine, then the second translation machine also hears the sound of the first translation machine, thereby realizing a normal communication between the first translation machine and the second translation machine;

in response to a reception of a voice signal from the second translation machine by the first translation machine, an other-side recognition thread starts to translate and translates into text message for displaying, then the first translation machine sees the call content of the second translation machine; and

in response to a determination that the first translation machine'"'"'s reply to the message is a text message, the first translation machine'"'"'s local recognition thread starts to translate the text message and sends a translation of the text message to the second translation machine through the TTS.

2. The smartphone-based telephone translation system in claim 1, wherein, in the first translation machine, the step of translating the voice signal to obtain a translation text comprises steps of:

performing a voice recognition on a voice message to obtain an original text; and

performing a text translation on the original text to obtain the translation text.

3. The smartphone-based telephone translation system in claim 2, wherein the first processor is configured to call the first program instructions for performing a step of recording a call time, a call duration, a call recording, an original text, and a translation text.

4. The smartphone-based telephone translation system in claim 2, wherein the first translation machine is configured to display the original text and the translation text.

5. The smartphone-based telephone translation system in claim 1, wherein the first processor is configured to call the first program instructions for performing steps of:

receiving a setting instruction from a user to open a call translation interface in response to a detection of an incoming call or dialing; and

displaying the call translation interface to start the call translation process in response to the detection of the incoming call or dialing.

6. The smartphone-based telephone translation system in claim 5, wherein the first processor is configured to call the first program instructions for performing steps of receiving an input instruction from the user to obtain a text message inputted by the user through a text input box set on the call translation interface.

7. The smartphone-based telephone translation system according to claim 1, wherein the first processor is configured to call the first program instructions for performing a step of receiving a switching instruction from a user to switch the call translation state to a normal call state.

8. The smartphone-based telephone translation system in claim 1, wherein the first processor is configured to call the first program instructions for performing steps of:

receiving a query instruction from a user to inquire a translated voice message or text message;

receiving a repeated transmission instruction from the user to select a translated voice signal or translated text for transmission to other side; and

receiving a repeated listening instruction from the user to listen to a translated voice signal.

9. The smartphone-based telephone translation system in claim 1, wherein the first processor is configured to call the first program instructions for performing a step of receiving a volume setting instruction from a user to adjust the first translation machine to mute or use a speaker.