Localized speech recognition with offload

• US 8,554,559 B1
• Filed: 01/21/2013
• Issued: 10/08/2013
• Est. Priority Date: 07/13/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Aleksic, Petar, Lei, Xin
• Primary Examiner(s): Harper, Vincent P
• Application Number:
  US13/746,115
• Time in Patent Office: 260 Days
• Field of Search: 704/235, 704/257, 704/275
• US Class Current: 704/235
• CPC Class Codes: G10L 15/07 to the speaker
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 21/00 Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility G10L19/00 takes precedence

Abstract

A local computing device may receive an utterance from a user device. In response to receiving the utterance, the local computing device may obtain a text string transcription of the utterance, and determine a response mode for the utterance. If the response mode is a text-based mode, the local computing device may provide the text string transcription to a target device. If the response mode is a non-text-based mode, the local computing device may convert the text string transcription into one or more commands from a command set supported by the target device, and provide the one or more commands to the target device.

Claims

1. A method comprising:

receiving, by a local computing device, an utterance from a user device, wherein the user device and the local computing device are part of a local network;

in response to receiving the utterance, obtaining a text string transcription of the utterance, and selecting a response mode for the utterance from among a text-based response mode and a non-text-based response mode, wherein obtaining the text string transcription of the utterance comprises transcribing the utterance into the text string transcription, wherein the text string transcription includes a representation of the utterance, and wherein transcribing the utterance into the text string transcription comprises (i) determining that the utterance fits a speaker adaptation profile, (ii) applying speaker adaptation parameters to the utterance, wherein the speaker adaptation parameters are associated with the speaker adaptation profile, and wherein the text string transcription is based on the speaker adaptation parameters, and (iii) updating the speaker adaptation parameters based at least on characteristics of the utterance;

if the response mode is a text-based response mode, providing the text string transcription to a target device;

if the response mode is a non-text-based response mode, (i) converting the text string transcription into one or more commands from a command set supported by the target device, and (ii) providing the one or more commands to the target device;

determining that the user device has speech recognition capability and supports speaker adaptation, and

based on the user device having speech recognition capability and supporting speaker adaptation, providing the updated speaker adaptation parameters to the user device.

2. The method of claim 1, wherein the target device is the user device.

3. The method of claim 1, further comprising:

receiving, by the local computing device, a second utterance from the user device;

in response to receiving the second utterance, obtaining a second text string transcription of the second utterance, wherein the second text string transcription is based on the updated speaker adaptation profile.

4. The method of claim 1, wherein obtaining the text string transcription of the utterance comprises:

providing the utterance to a remote computing device; and

receiving the text string transcription from the remote computing device, wherein the text string transcription includes a representation of the utterance.

5. The method of claim 4, wherein the remote computing device is not part of the local network.

6. The method of claim 1, wherein the non-text-based response mode is a command-based response mode.

7. The method of claim 1, wherein selecting the response mode for the utterance comprises:

looking up a profile of the target device in a database accessible to the local computing device; and

selecting the response mode for the utterance based of the profile of the target device.

8. The method of claim 1, wherein selecting the response mode for the utterance comprises selecting the response mode for the utterance based on a message sent by the user device, the message containing at least part of the utterance.

9. An article of manufacture including a computer-readable storage medium, having stored thereon program instructions that, upon execution by a local computing device, cause the local computing device to perform operations comprising:

receiving an utterance from a user device, wherein the user device and the local computing device are part of a local network;

in response to receiving the utterance, obtaining a text string transcription of the utterance, and selecting a response mode for the utterance from among a text-based response mode and a non-text-based response mode, wherein obtaining the text string transcription of the utterance comprises transcribing the utterance into the text string transcription, wherein the text string transcription includes a representation of the utterance, and wherein transcribing the utterance into the text string transcription comprises (i) determining that the utterance fits a speaker adaptation profile, (ii) applying speaker adaptation parameters to the utterance, wherein the speaker adaptation parameters are associated with the speaker adaptation profile, and wherein the text string transcription is based on the speaker adaptation parameters, and (iii) updating the speaker adaptation parameters based at least on characteristics of the utterance;

if the response mode is a text-based response mode, providing the text string transcription to a target device;

if the response mode is a non-text-based response mode, (i) converting the text string transcription into one or more commands from a command set supported by the target device, and (ii) providing the one or more commands to the target device;

determining that the user device has speech recognition capability and supports speaker adaptation, and

based on the user device having speech recognition capability and supporting speaker adaptation, providing the updated speaker adaptation parameters to the user device.

10. The article of manufacture of claim 9, wherein obtaining the text string transcription of the utterance comprises:

providing the utterance to a remote computing device; and

receiving the text string transcription from the remote computing device, wherein the text string transcription includes a representation of the utterance.

11. A method comprising:

receiving, by a local computing device, a first utterance from a first user device;

in response to receiving the first utterance, obtaining a first text string transcription of the first utterance, wherein obtaining the first text string transcription of the first utterance comprises transcribing, by an automatic speech recognition (ASR) module of the local computing device, the first utterance into the first text string transcription, wherein the first text string transcription includes a representation of the first utterance, and wherein transcribing the first utterance into the first text string transcription comprises (i) determining that the first utterance fits a speaker adaptation profile, (ii) applying speaker adaptation parameters to the utterance, wherein the speaker adaptation parameters are associated with the speaker adaptation profile, and wherein the first text string transcription is based on the speaker adaptation parameters, and (iii) updating the speaker adaptation parameters based at least on characteristics of the first utterance;

determining that the first user device seeks a textual representation of the first utterance;

in response to determining that the first user device seeks the textual representation of the first utterance, transmitting the first text string transcription to the first user device;

determining that the first user device has speech recognition capability and supports speaker adaptation;

based on the first user device having speech recognition capability and supporting speaker adaptation, providing the updated speaker adaptation parameters to the first user device;

receiving, by the local computing device, a second utterance from a second user device, wherein the first user device, the second user device, and the local computing device are part of a local network;

in response to receiving the second utterance, obtaining a second text string transcription of the second utterance;

determining that the second user device seeks a command-based representation of the second utterance; and

in response to determining that the second user device seeks the command-based representation of the second utterance, (i) converting the second text string transcription into one or more commands in a command set supported by the second user device, and (iii) transmitting the one or more commands to the second user device.

12. The method of claim 11, wherein obtaining the second text string transcription of the second utterance comprises transcribing, by the ASR module of the local computing device, the second utterance into the second text string transcription, wherein the second text string transcription includes a representation of the second utterance, and wherein transcribing the second utterance into the second text string transcription comprises:

determining that the second utterance fits the speaker adaptation profile;

applying the updated speaker adaptation parameters to the second utterance, wherein the second text string transcription is based on the updated speaker adaptation parameters; and

further updating the speaker adaptation parameters based at least on characteristics of the second utterance.

13. The method of claim 11, wherein obtaining the first text string transcription of the first utterance comprises:

providing the first utterance to a remote computing device; and

receiving the first text string transcription from the remote computing device, wherein the first text string transcription includes a representation of the utterance.

14. The method of claim 13, wherein the remote computing device is not part of the local network.