CONTEXT-SENSITIVE DYNAMIC UPDATE OF VOICE TO TEXT MODEL IN A VOICE-ENABLED ELECTRONIC DEVICE

• US 20160351194A1
• Filed: 05/27/2015
• Published: 12/01/2016
• Est. Priority Date: 05/27/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Gao, Yuli, Sung, Sangsoo, Murugesan, Prathab
• Granted Patent: US 9,966,073 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G10L 15/065 Adaptation
• CPC Class Codes: G10L 15/083 Recognition networks G10L15/142, G10L15/16 take precedence
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/228 of application context

Abstract

A voice to text model used by a voice-enabled electronic device is dynamically and in a context-sensitive manner updated to facilitate recognition of entities that potentially may be spoken by a user in a voice input directed to the voice-enabled electronic device. The dynamic update to the voice to text model may be performed, for example, based upon processing of a first portion of a voice input, e.g., based upon detection of a particular type of voice action, and may be targeted to facilitate the recognition of entities that may occur in a later portion of the same voice input, e.g., entities that are particularly relevant to one or more parameters associated with a detected type of voice action.

Claims

1. A method, comprising:

receiving a voice input with a voice-enabled electronic device; and

in the voice-enabled electronic device, and responsive to receiving at least a portion of the voice input;

performing local processing of the at least a portion of the voice input to dynamically build at least a portion of a voice action prior to completely receiving the voice input with the voice-enabled electronic device;

determining during the local processing whether the voice action is associated with a context sensitive parameter; and

in response to a determination that the voice action is associated with the context sensitive parameter, initiating a dynamic update to a local voice to text model used by the voice-enabled electronic device prior to completing the voice action to facilitate recognition of a plurality of context sensitive entities associated with the context sensitive parameter.

2. The method of claim 1, wherein performing the local processing includes:

converting a digital audio signal of the voice input to text using a streaming voice to text module of the voice-enabled electronic device, wherein the streaming voice to text module dynamically generates a plurality of text tokens from the digital audio signal; and

dynamically building the portion of the voice action from at least a portion of the plurality of text tokens using a streaming semantic processor of the voice-enabled electronic device.

3. The method of claim 2, wherein determining whether the voice action is associated with the context sensitive parameter is performed by the streaming semantic processor, and wherein initiating the dynamic update to the local voice to text model includes communicating data from the streaming semantic processor to the streaming voice to text module to initiate the dynamic update of the local voice to text model.

4. The method of claim 1, wherein the local voice to text model comprises at least one decoding graph, and wherein initiating the dynamic update of the local voice to text model includes adding a decoding path to the at least one decoding graph corresponding to each of the plurality of context sensitive entities.

5. The method of claim 1, further comprising, in response to a determination that the voice action is associated with the context sensitive parameter, prefetching from an online service voice to text model update data associated with the plurality of context sensitive entities, wherein initiating the dynamic update of the local voice to text model includes communicating the prefetched voice to text model update data to dynamically update the local voice to text model.

6. The method of claim 1, wherein determining during the local processing whether the voice action is associated with a context sensitive parameter includes determining whether the voice action is a request to play a media item, wherein the context sensitive parameter includes a media item identifier for use in identifying the media item, and wherein the plurality of context sensitive entities identify a plurality of media items playable by the voice-enabled electronic device.

7. The method of claim 1, wherein determining during the local processing whether the voice action is associated with a context sensitive parameter includes determining whether the voice action is a request to communicate with a contact, wherein the context sensitive parameter includes a contact identifier for use in initiating a communication with the contact, and wherein the plurality of context sensitive entities identify a plurality of contacts accessible by the voice-enabled electronic device.

8. The method of claim 1, wherein the context sensitive parameter is a location-dependent parameter, and wherein the plurality of context sensitive entities identify a plurality of points of interest disposed in proximity to a predetermined location.

9. The method of claim 8, wherein the predetermined location comprises a current location of the voice-enabled electronic device.

10. The method of claim 9, further comprising, in response to a determination that the voice action is associated with the context sensitive parameter, communicating the current location to an online service and prefetching from the online service voice to text model update data associated with the plurality of context sensitive entities.

11. A method, comprising:

receiving a voice input including first and second portions with a voice-enabled electronic device, the voice input associated with a voice action having a context sensitive parameter;

performing voice to text conversion locally in the voice-enabled electronic device on the first portion of the voice input using a local voice to text model to generate text for the first portion of the voice input;

dynamically updating the local voice to text model after generating the text for the first portion of the voice input to facilitate recognition of a plurality of context sensitive entities associated with the context sensitive parameter for the voice action associated with the voice input; and

performing voice to text conversion locally in the voice-enabled electronic device on the second portion of the voice input using the dynamically updated local voice to text model to generate text for the second portion of the voice input.

12. The method of claim 11, wherein performing the voice to text conversion includes converting a digital audio signal of the voice input to text using a streaming voice to text module of the voice-enabled electronic device, wherein the streaming voice to text module dynamically generates a plurality of text tokens from the digital audio signal, the method further comprising dynamically building at least a portion of the voice action prior to completely receiving the voice input with the voice-enabled electronic device from at least a portion of the plurality of text tokens using a streaming semantic processor of the voice-enabled electronic device.

13. The method of claim 12, wherein dynamically updating the local voice to text model is initiated by the streaming semantic processor in response to determining that the voice action is associated with the context sensitive parameter.

14. An apparatus including memory and one or more processors operable to execute instructions stored in the memory, comprising instructions to:

receive a voice input with a voice-enabled electronic device; and

in the voice-enabled electronic device, and responsive to receiving at least a portion of the voice input;

perform local processing of the at least a portion of the voice input to dynamically build at least a portion of a voice action prior to completely receiving the voice input with the voice-enabled electronic device;

determine during the local processing whether the voice action is associated with a context sensitive parameter; and

in response to a determination that the voice action is associated with the context sensitive parameter, initiate a dynamic update to a local voice to text model used by the voice-enabled electronic device prior to completing the voice action to facilitate recognition of a plurality of context sensitive entities associated with the context sensitive parameter.

15. The apparatus of claim 14, wherein the instructions include:

first instructions implementing a streaming voice to text module that converts a digital audio signal of the voice input to text, wherein the first instructions dynamically generate a plurality of text tokens from the digital audio signal; and

second instructions implementing a streaming semantic processor that dynamically builds the portion of the voice action from at least a portion of the plurality of text tokens.

16. The apparatus of claim 15, wherein the instructions that implement the streaming semantic processor determine whether the voice action is associated with the context sensitive parameter, and wherein the instructions that implement the streaming semantic processor communicate data from the streaming semantic processor to the streaming voice to text module to initiate the dynamic update of the local voice to text model.

17. The apparatus of claim 15, further comprising instructions that, in response to a determination that the voice action is associated with the context sensitive parameter, prefetch from an online service voice to text model update data associated with the plurality of context sensitive entities, wherein the instructions that initiate the dynamic update of the local voice to text model communicate the prefetched voice to text model update data to dynamically update the local voice to text model.

18. The apparatus of claim 15, wherein the instructions that determine during the local processing whether the voice action is associated with a context sensitive parameter determine whether the voice action is a request to play a media item, wherein the context sensitive parameter includes media item data for use in identifying the media item, and wherein the plurality of context sensitive entities includes identifiers for a plurality of media items playable by the voice-enabled electronic device.

19. The apparatus of claim 15, wherein the instructions that determine during the local processing whether the voice action is associated with a context sensitive parameter determine whether the voice action is a request to communicate with a contact, wherein the context sensitive parameter includes contact data for use in initiating a communication with the contact, and wherein the plurality of context sensitive entities includes identifiers for a plurality of contacts accessible by the voice-enabled electronic device.

20. The apparatus of claim 15, wherein the context sensitive parameter is a location-dependent parameter, wherein the plurality of context sensitive entities includes identifiers for a plurality of points of interest disposed in proximity to a predetermined location, the apparatus further comprising instructions that, in response to a determination that the voice action is associated with the context sensitive parameter, communicate the predetermined location to an online service and prefetch from the online service voice to text model update data associated with the identifiers for the plurality of points of interest disposed in proximity to the predetermined location.

21. A non-transitory computer readable storage medium storing computer instructions executable by one or more processors to perform a method comprising:

receiving a voice input with a voice-enabled electronic device; and

in the voice-enabled electronic device, and responsive to receiving at least a portion of the voice input;

performing local processing of the at least a portion of the voice input to dynamically build at least a portion of a voice action prior to completely receiving the voice input with the voice-enabled electronic device;

determining during the local processing whether the voice action is associated with a context sensitive parameter; and

in response to a determination that the voice action is associated with the context sensitive parameter, initiating a dynamic update to a local voice to text model used by the voice-enabled electronic device prior to completing the voice action to facilitate recognition of a plurality of context sensitive entities associated with the context sensitive parameter.