USER INTERFACE FOR REALTIME LANGUAGE TRANSLATION

• US 20180276203A1
• Filed: 05/08/2018
• Published: 09/27/2018
• Est. Priority Date: 11/08/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Cuthbert, Alexander Jay, Goyal, Sunny, Gaba, Matthew Morton, Estelle, Joshua J., Seno, Masakazu
• Granted Patent: US 10,496,759 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 25/48 specially adapted for particular use

Abstract

A language translation application on a user device includes a user interface that provides relevant textual and graphical feedback mechanisms associated with various states of voice input and translated speech.

Claims

1. (canceled)

2. A computer-implemented method comprising:

providing, by a mobile device, a user interface for a real-time translation application that receives spoken inputs in a first language outputs and that generates synthesized speech representations of translations of the spoken inputs in a second language, wherein the user interface includes;

a first language region for displaying transcriptions of the spoken inputs in the first language,

a second language region for displaying translations of the spoken inputs in the second language, and

an indicator region for displaying an indicator that indicates that (i) a microphone of the mobile device is preparing to receive spoken inputs but is not yet prepared to receive spoken inputs, or (ii) a speaker of the mobile device is preparing to output synthesized speech representations in the second language but is not yet outputting synthesized speech representations in the second language;

determining, by the mobile device, a state of the real-time translation application; and

in response to determining the state of the real-time translation application, updating, by the mobile device, the user interface to display the indicator that indicates that (i) the microphone of the mobile device is preparing to receive spoken inputs but is not yet prepared to receive spoken inputs, or (ii) the speaker of the mobile device is preparing to output synthesized speech representations in the second language but is not yet outputting synthesized speech representations in the second language.

3. The method of claim 2, comprising:

receiving a particular spoken input; and

updating (i) the first language region of the user interface to display a transcription of the particular spoken input in a first language, and (ii) the second language region of the user interface to display a translation of the particular spoken input in a second language.

4. The method of claim 2, wherein the mobile device includes an automated speech recognizer, and an automated language translator.

5. The method of claim 4, wherein the automated speech recognizer comprises a continuous speech recognizer.

6. The method of claim 2, wherein the indicator comprises a microphone icon or a speaker icon.

7. The method of claim 2, wherein the indicator is user-selectable to manually activate or deactivate the speaker or the microphone.

8. The method of claim 2, wherein the first language region is further for displaying translations of spoken inputs in the second language, in the first language.

9. A system comprising one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:

providing, by a mobile device, a user interface for a real-time translation application that receives spoken inputs in a first language outputs and that generates synthesized speech representations of translations of the spoken inputs in a second language, wherein the user interface includes;

a first language region for displaying transcriptions of the spoken inputs in the first language,

a second language region for displaying translations of the spoken inputs in the second language, and

an indicator region for displaying an indicator that indicates that (i) a microphone of the mobile device is preparing to receive spoken inputs but is not yet prepared to receive spoken inputs, or (ii) a speaker of the mobile device is preparing to output synthesized speech representations in the second language but is not yet outputting synthesized speech representations in the second language;

determining, by the mobile device, a state of the real-time translation application; and

in response to determining the state of the real-time translation application, updating, by the mobile device, the user interface to display the indicator that indicates that (i) the microphone of the mobile device is preparing to receive spoken inputs but is not yet prepared to receive spoken inputs, or (ii) the speaker of the mobile device is preparing to output synthesized speech representations in the second language but is not yet outputting synthesized speech representations in the second language.

10. The system of claim 9, wherein the operations comprise:

receiving a particular spoken input; and

updating (i) the first language region of the user interface to display a transcription of the particular spoken input in a first language, and (ii) the second language region of the user interface to display a translation of the particular spoken input in a second language.

11. The system of claim 9, wherein the mobile device includes an automated speech recognizer, and an automated language translator.

12. The system of claim 11, wherein the automated speech recognizer comprises a continuous speech recognizer.

13. The system of claim 9, wherein the indicator comprises a microphone icon or a speaker icon.

14. The system of claim 9, wherein the indicator is user-selectable to manually activate or deactivate the speaker or the microphone.

15. The system of claim 9, wherein the first language region is further for displaying translations of spoken inputs in the second language, in the first language.

16. A computer-readable storage device storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

providing, by a mobile device, a user interface for a real-time translation application that receives spoken inputs in a first language outputs and that generates synthesized speech representations of translations of the spoken inputs in a second language, wherein the user interface includes;

a first language region for displaying transcriptions of the spoken inputs in the first language,

a second language region for displaying translations of the spoken inputs in the second language, and

an indicator region for displaying an indicator that indicates that (i) a microphone of the mobile device is preparing to receive spoken inputs but is not yet prepared to receive spoken inputs, or (ii) a speaker of the mobile device is preparing to output synthesized speech representations in the second language but is not yet outputting synthesized speech representations in the second language;

determining, by the mobile device, a state of the real-time translation application; and

in response to determining the state of the real-time translation application, updating, by the mobile device, the user interface to display the indicator that indicates that (i) the microphone of the mobile device is preparing to receive spoken inputs but is not yet prepared to receive spoken inputs, or (ii) the speaker of the mobile device is preparing to output synthesized speech representations in the second language but is not yet outputting synthesized speech representations in the second language.

17. The device of claim 16, wherein the operations comprise:

receiving a particular spoken input; and

updating (i) the first language region of the user interface to display a transcription of the particular spoken input in a first language, and (ii) the second language region of the user interface to display a translation of the particular spoken input in a second language.

18. The device of claim 16, wherein the mobile device includes an automated speech recognizer, and an automated language translator.

19. The device of claim 18, wherein the automated speech recognizer comprises a continuous speech recognizer.

20. The device of claim 16, wherein the indicator comprises a microphone icon or a speaker icon.

21. The device of claim 16, wherein the indicator is user-selectable to manually activate or deactivate the speaker or the microphone.