Automatic speech recognition techniques for multiple languages

• US 9,953,631 B1
• Filed: 05/07/2015
• Issued: 04/24/2018
• Est. Priority Date: 05/07/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Cuthbert, Alexander Jay, Seno, Masakazu, Turovsky, Barak
• Primary Examiner(s): Mishra, Richa
• Application Number:
  US14/706,243
• Time in Patent Office: 1,083 Days
• Field of Search: None
• CPC Class Codes: G06F 40/263 Language identification
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G10L 15/005 Language recognition
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

Computer-implemented techniques can include obtaining languages for automatic speech recognition, the languages being associated with a plurality of users and receiving an input indicative of a request to provide speech input. In response to receiving the input, a first indication that automatic speech recognition (ASR) is being performed in each of the plurality of languages can be displayed. A speech input from one of the plurality of users can be received and ASR can identify one of the languages associated with the speech input to obtain a detected language. In response to obtaining the detected language, a second indication that ASR is being performed in the detected language can be displayed. The speech input can be transcribed and translated to another one of the plurality of languages to obtain a translated text that can be displayed.

Claims

1. A computer-implemented method, comprising:

obtaining, at a computing device having one or more processors, a plurality of languages for automatic speech recognition, the plurality of languages being associated with a plurality of users;

receiving, at the computing device, an input indicative of a request to provide speech input;

receiving, by the computing device, a speech input from one of the plurality of users;

in response to receiving the input;

displaying, by the computing device, a first indication that the computing device is performing automatic speech recognition in two or more of the plurality of languages, the first indication comprising highlighted icons displaying the two or more of the plurality of languages, and

performing, by the computing device, automatic speech recognition on the speech input in the two or more of the plurality of languages to identify one of the plurality of languages associated with the speech input to obtain a detected language;

obtaining, by the computing device, a transcription of the speech input to obtain a text in the detected language;

in response to obtaining the text, displaying, by the computing device, the text in a first area on a display of the computing device;

obtaining, by the computing device, a translation of the text from the detected language to another one of the plurality of languages to obtain a translated text; and

displaying, by the computing device, the translated text in a separate second area on the display,

wherein the highlighted icons displaying the two or more of the plurality of languages are displayed in a separate third area of the display that is distinct from the first and second areas of the display.

2. The computer-implemented method of claim 1, further comprising in response to obtaining the detected language, displaying, by the computing device, a second indication that the computing device is performing automatic speech recognition in the detected language.

3. The computer-implemented method of claim 2, wherein the second indication is a highlighted icon displaying the detected language and non-highlighted icons displaying a remainder of the two or more of the plurality of languages, respectively.

4. The computer-implemented method of claim 3, further comprising displaying, by the computing device, a third indication of whether automatic speech recognition is currently being performed, wherein the third indication is a highlighted icon from when the request is received until a completion of the speech input, and wherein the third indication is a non-highlighted icon upon the completion of the speech input.

5. The computer-implemented method of claim 1, wherein the text is displayed in at least one of a first color and a first style, and wherein the translated text is displayed in at least one of a different second color and a different second style.

6. A computing device having one or more processors configured to perform operations comprising:

obtaining a plurality of languages for automatic speech recognition, the plurality of languages being associated with a plurality of users;

receiving an input indicative of a request to provide speech input;

receiving a speech input from one of the plurality of users;

in response to receiving the input;

displaying a first indication that the computing device is performing automatic speech recognition in two or more of the plurality of languages, the first indication comprising highlighted icons displaying the two or more of the plurality of languages,

performing automatic speech recognition on the speech input in the two or more of the plurality of languages to identify one of the plurality of languages associated with the speech input to obtain a detected language, and

in response to obtaining the detected language, displaying a second indication that the computing device is performing automatic speech recognition in the detected language, wherein the second indication is a highlighted icon displaying the detected language and non-highlighted icons displaying a remainder of the two or more of the plurality of languages, respectively;

obtaining a transcription of the speech input to obtain a text in the detected language;

obtaining a translation of the text from the detected language to another one of the plurality of languages to obtain a translated text; and

displaying the translated text.

7. The computing device of claim 6, wherein the operations further comprise displaying a third indication of whether automatic speech recognition is currently being performed, wherein the third indication is a highlighted icon from when the request is received until a completion of the speech input, and wherein the third indication is a non-highlighted icon upon the completion of the speech input.

8. The computing device of claim 6, wherein the operations further comprise displaying the text in the detected language in response to obtaining the text, wherein the text is displayed in a first area on a display of the computing device, and wherein the translated text is displayed in a separate second area on the display.

9. The computing device of claim 8, wherein the text is displayed in at least one of a first color and a first style, and wherein the translated text is displayed in at least one of a different second color and a different second style.

10. A non-transitory computer-readable medium having instructions stored thereon that, when executed by one or more processors of a computing device, cause the computing device to perform operations comprising:

obtaining a plurality of languages for automatic speech recognition, the plurality of languages being associated with a plurality of users;

receiving an input indicative of a request to provide speech input;

receiving a speech input from one of the plurality of users;

in response to receiving the input;

displaying a first indication that the computing device is performing automatic speech recognition in two or more of the plurality of languages, the first indication comprising highlighted icons displaying the two or more of the plurality of languages,

performing automatic speech recognition on the speech input in the two or more of the plurality of languages to identify one of the plurality of languages associated with the speech input to obtain a detected language, and

in response to obtaining the detected language, displaying a second indication that the computing device is performing automatic speech recognition in the detected language, wherein the second indication is a highlighted icon displaying the detected language and non-highlighted icons displaying a remainder of the two or more of the plurality of languages, respectively;

obtaining a transcription of the speech input to obtain a text in the detected language;

obtaining a translation of the text from the detected language to another one of the plurality of languages to obtain a translated text; and

displaying the translated text.

11. The computer-readable medium of claim 10, wherein the operations further comprise displaying a third indication of whether automatic speech recognition is currently being performed, wherein the third indication is a highlighted icon from when the request is received until a completion of the speech input, and wherein the third indication is a non-highlighted icon upon the completion of the speech input.

12. The computer-readable medium of claim 10, wherein the operations further comprise displaying the text in the detected language in response to obtaining the text, wherein the text is displayed in a first area on a display of the computing device, wherein the translated text is displayed in a separate second area on the display, wherein the text is displayed in at least one of a first color and a first style, and wherein the translated text is displayed in at least one of a different second color and a different second style.