System and method for managing models for embedded speech and language processing

• US 9,773,498 B2
• Filed: 12/01/2016
• Issued: 09/26/2017
• Est. Priority Date: 10/28/2013
• Status: Active Grant

• Current Assignee: Hyundai Motor Company (Hyundai Motor Group), Kia Corp.
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Stern, Benjamin J., Bocchieri, Enrico Luigi, Conkie, Alistair D., Giulianelli, Danilo
• Primary Examiner(s): Armstrong, Angela A
• Application Number:
  US15/366,292
• Publication Number:
  US 20170084270A1
• Time in Patent Office: 299 Days
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/228 of application context

Abstract

Disclosed herein are systems, methods, and computer-readable storage devices for fetching speech processing models based on context changes in advance of speech requests using the speech processing models. An example local device configured to practice the method, having a local speech processor, and having access to remote speech models, detects a change in context. The change in context can be based on geographical location, language translation, speech in a different language, user language settings, installing or removing an app, and so forth. The local device can determine a speech processing model that is likely to be needed based on the change in context, and that is not stored on the local device. Independently of an explicit request to process speech, the local device can retrieve, from a remote server, the speech processing model for use on the mobile device.

Claims

1. A method comprising:

detecting, at a mobile device, a change in geographic location of the mobile device to yield a new context of a speech dialogue;

calculating a likelihood score for each speech model in a plurality of speech models on a remote server, wherein the likelihood score indicates a compatibility of each speech model to recognize speech in the speech dialogue in the new context to yield likelihood scores;

selecting a speech model from the plurality of speech models according to the likelihood scores;

calculating local likelihood scores of local speech models stored in a local storage of the mobile device, wherein the local likelihood scores indicate a probability of each local speech model to recognize the speech in the speech dialogue in the new context to yield a respective scored local speech model; and

removing the respective scored local speech model having a lowest likelihood score of the local likelihood scores until a sufficient space for the speech model from the remote server is made available on the mobile device.

2. The method of claim 1, wherein the likelihood scores are calculated according to one or more of a certainty of future use, an urgency of future use, and a reusability of each speech model in the plurality of speech models.

3. The method of claim 1, wherein the likelihood scores are weighted according to one or more of historical data comprising contextual history and speech model usage.

4. The method of claim 1, wherein the new context is further based on one of language translation, speech in a different language, user language settings, available local storage, installation of an application on the mobile device, and removal of the application on the mobile device.

5. The method of claim 1, further comprising:

comparing network characteristics to a priority for the speech model to determine when to retrieve the speech model.

6. The method of claim 1, further comprising:

determining that an existing speech model stored on the mobile device has a lower likelihood score than the speech model; and

removing the existing speech model from local storage on the mobile device.

7. The method of claim 1, further comprising identifying the plurality of speech models on the remote server.

8. A system comprising:

a processor; and

a computer-readable storage medium having instructions stored which, when executed by the processor, cause the processor to perform operations comprising;

detecting, at a mobile device, a change in geographic location of the mobile device to yield a new context of a speech dialogue;

calculating a likelihood score for each speech model in a plurality of speech models on a remote server, wherein the likelihood score indicates a compatibility of each speech model to recognize speech in the speech dialogue in the new context to yield likelihood scores;

selecting a speech model from the plurality of speech models according to the likelihood scores;

calculating local likelihood scores of local speech models stored in a local storage of the mobile device, wherein the local likelihood scores indicate a probability of each local speech model to recognize the speech in the speech dialogue in the new context to yield a respective scored local speech model; and

removing the respective scored local speech model having a lowest likelihood score of the local likelihood scores until a sufficient space for the speech model from the remote server is made available on the mobile device.

9. The system of claim 8, wherein the new context is based on one or more of unused storage space on the mobile device crossing a threshold, language translation, speech in a different language, user language settings, installing an application on the mobile device, and removing the application on the mobile device.

10. The system of claim 8, wherein the likelihood scores are calculated according to one or more of a certainty of future use, an urgency of future use, and a reusability of each speech model in the plurality of speech models.

11. The system of claim 8, wherein the likelihood scores are weighted according to historical data comprising one or more of contextual history and speech model usage.

12. The system of claim 8, the computer-readable storage medium having additional instructions stored which, when executed by the processor, cause the processor to perform operations comprising:

determining that an existing speech model stored on the mobile device has a lower likelihood score than the speech model; and

removing the existing speech model from local storage on the mobile device.

13. The system of claim 8, the computer-readable storage medium having additional instructions stored which, when executed by the processor, cause the processor to perform operations comprising:

determining that a plurality of previously stored speech models on the mobile device have likelihood scores below a threshold likelihood score, to yield below-threshold speech models; and

removing the below-threshold speech models from the mobile device.

14. The system of claim 13, wherein the removing of the below-threshold speech models from the mobile device occurs only until a threshold amount of storage space on the mobile device is freed.

15. A computer-readable storage device having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising:

detecting, at a mobile device, a change in geographic location of the mobile device to yield a new context of a speech dialogue;

calculating a likelihood score for each speech model in a plurality of speech models on a remote server, wherein the likelihood score indicates a compatibility of each speech model to recognize speech in the speech dialogue in the new context to yield likelihood scores;

selecting a speech model from the plurality of speech models according to the likelihood scores;

calculating local likelihood scores of local speech models stored in a local storage of the mobile device, wherein the local likelihood scores indicate a probability of each local speech model to recognize the speech in the speech dialogue in the new context to yield a respective scored local speech model; and

removing the respective scored local speech model having a lowest likelihood score of the local likelihood scores until a sufficient space for the speech model from the remote server is made available on the mobile device.

16. The computer-readable storage device of claim 15, wherein the likelihood scores are calculated according to one or more of a certainty of future use, an urgency of future use, and a reusability of each speech model in the plurality of speech models.

17. The computer-readable storage device of claim 15, wherein the likelihood scores are weighted according to one or more of historical data comprising contextual history and speech model usage.

18. The computer-readable storage device of claim 15, wherein the new context is based on one of language translation, speech in a different language, user language settings, available local storage, installing an application on the mobile device, and removing the application on the mobile device.

19. The computer-readable storage device of claim 15, having additional instructions stored which, when executed by the computing device, cause the computing device to perform operations comprising:

comparing network characteristics to a priority for the speech model to determine when to retrieve the speech model.

20. The computer-readable storage device of claim 15, the computer-readable storage device having additional instructions stored which, when executed by the computing device, cause the computing device to perform operations comprising:

determining that a plurality of previously stored speech models on the mobile device have likelihood scores below a threshold likelihood score, to yield below-threshold speech models; and

removing the below-threshold speech models from the mobile device.