USER INTERFACE FOR REALTIME LANGUAGE TRANSLATION

• US 20150051898A1
• Filed: 11/12/2013
• Published: 02/19/2015
• Est. Priority Date: 08/14/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Cuthbert, Alexander J., Estelle, Joshua J., Hughes, Macduff Richard, Goyal, Sunny, Jiang, Minqi Sebastian
• Granted Patent: US 9,355,094 B2
• Time in Patent Office: Days
• US Class Current: 704/3
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/263 Language identification
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G10L 13/086 Detection of language
• CPC Class Codes: G10L 15/005 Language recognition
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

A data processing apparatus receives data indicating a movement of a client device by a first user. The apparatus determines that the movement of the client device is a delimiter motion for switching between a first mode, in which the client device is configured to (i) provide a first interface for a first user speaking in a first language and (ii) perform speech recognition of the first language, and a second mode, in which the client device is configured to (i) provide a second interface for a second user speaking in a second language and (ii) perform speech recognition of the second language, the second interface being different from the first interface. Based on determining that the movement is a delimiter motion, the apparatus switches between the first mode and the second mode without the second user physically interacting with the client device.

Claims

1. A computer-implemented method comprising:

receiving data indicating a movement of a client device by a first user;

determining that the movement of the client device is a delimiter motion for switching between a first mode, in which the client device is configured to (i) provide a first interface for a first user speaking in a first language and (ii) perform speech recognition of the first language, and a second mode, in which the client device is configured to (i) provide a second interface for a second user speaking in a second language and (ii) perform speech recognition of the second language, the second interface being different from the first interface; and

based on determining that the movement is a delimiter motion, switching between the first mode and the second mode without the second user physically interacting with the client device.

2. The method of claim 1, wherein the movement of the client device by the first user comprises rotation of the client device by the first user.

3. The method of claim 1, wherein the first interface for the first user comprises a transcription of an utterance of the first user and a translation of the utterance into the second language.

4. The method of claim 1, wherein the second interface for the second user comprises a full screen translation of an utterance by the first user into the second language.

5. The method of claim 1, further comprising, while the client device is in the first mode,

receiving an utterance of the first user in the first language;

and

outputting an audio signal encoding a translation of the utterance into the second language.

6. The method of claim 5, further comprising, while the client device is in the first mode,

translating, at the client device, the utterance of the first user into the second language.

7. The method of claim 1, further comprising, while the client device is in the second mode,

receiving an utterance of the second user in the second language;

and

outputting an audio signal encoding a translation of the utterance into the first language.

8. A client device comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

receiving data indicating a movement of a client device by a first user;

determining that the movement of the client device is a delimiter motion for switching between a first mode, in which the client device is configured to (i) provide a first interface for a first user speaking in a first language and (ii) perform speech recognition of the first language, and a second mode, in which the client device is configured to (i) provide a second interface for a second user speaking in a second language and (ii) perform speech recognition of the second language, the second interface being different from the first interface; and

based on determining that the movement is a delimiter motion, switching between the first mode and the second mode without the second user physically interacting with the client device.

9. The client device of claim 8, wherein the movement of the client device by the first user comprises rotation of the client device by the first user.

10. The client device of claim 8, wherein the first interface for the first user comprises a transcription of an utterance of the first user and a translation of the utterance into the second language.

11. The client device of claim 8, wherein the second interface for the second user comprises a full screen translation of an utterance by the first user into the second language.

12. The client device of claim 8, wherein the operations further comprise, while the client device is in the first mode,

receiving an utterance of the first user in the first language;

and

outputting an audio signal encoding a translation of the utterance into the second language.

13. The client device of claim 12, wherein the operations further comprise, while the client device is in the first mode,

translating, at the client device, the utterance of the first user into the second language.

14. The client device of claim 8, wherein the operations further comprise, while the client device is in the second mode,

receiving an utterance of the second user in the second language;

and

outputting an audio signal encoding a translation of the utterance into the first language.

15. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

receiving data indicating a movement of a client device by a first user;

determining that the movement of the client device is a delimiter motion for switching between a first mode, in which the client device is configured to (i) provide a first interface for a first user speaking in a first language and (ii) perform speech recognition of the first language, and a second mode, in which the client device is configured to (i) provide a second interface for a second user speaking in a second language and (ii) perform speech recognition of the second language, the second interface being different from the first interface; and

based on determining that the movement is a delimiter motion, switching between the first mode and the second mode without the second user physically interacting with the client device.

16. The computer-readable medium of claim 15, wherein the movement of the client device by the first user comprises rotation of the client device by the first user.

17. The computer-readable medium of claim 15, wherein the first interface for the first user comprises a transcription of an utterance of the first user and a translation of the utterance into the second language.

18. The computer-readable medium of claim 15, wherein the second interface for the second user comprises a full screen translation of an utterance by the first user into the second language.

19. The computer-readable medium of claim 15, wherein the operations further comprise, while the client device is in the first mode,

receiving an utterance of the first user in the first language;

and

outputting an audio signal encoding a translation of the utterance into the second language.

20. The computer-readable medium of claim 19, wherein the operations further comprise, while the client device is in the first mode,

translating, at the client device, the utterance of the first user into the second language.