COMPUTER IMPLEMENTED METHODS AND APPARATUS FOR SELECTIVELY INTERACTING WITH A SERVER TO BUILD A LOCAL DICTATION DATABASE FOR SPEECH RECOGNITION AT A DEVICE

• US 20140006028A1
• Filed: 07/02/2013
• Published: 01/02/2014
• Est. Priority Date: 07/02/2012
• Status: Active Grant

• Current Assignee: Salesforce.com, Inc.
• Original Assignee: Salesforce.com, Inc.
• Inventors: Hu, Minzhi
• Granted Patent: US 9,715,879 B2
• Time in Patent Office: Days
• US Class Current: 704/251
• CPC Class Codes: G10L 15/10 using distance or distortion measures between unknown speech and reference templates
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 17/04 Training, enrolment or model building
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

Disclosed are methods, apparatus, systems, and computer-readable storage media for selectively interacting with a server to build a local dictation database for speech recognition at a device. In some implementations, a computing device receives an audio sample. The computing device may determine that the received audio sample does not match any of one or more existing audio samples stored in the local dictation database of the computing device. The received audio sample may be transmitted to a remote server for detection of one or more words indicated by the received audio sample. The computing device may receive data identifying the one or more words, and update the local dictation database to store the received audio sample in association with the one or more words.

Claims

1. A method for selectively interacting with a server to build a local dictation database for speech recognition at a device, the method comprising:

receiving, at a computing device associated with a user, a first audio sample;

determining, at the computing device, that the first audio sample does not match any of one or more audio samples stored in a local dictation database of the computing device;

transmitting the first audio sample to a remote server for detection of one or more words indicated by the first audio sample;

receiving data identifying the one or more words from the remote server; and

updating the local dictation database to store the first audio sample in association with the one or more words.

2. The method of claim 1, the method further comprising:

determining that the one or more words are associated with a first executable action of the computing device.

3. The method of claim 2, wherein one or more executable actions of the computing device are defined by and executable by an application installed on the computing device.

4. The method of claim 2, wherein one or more executable actions of the computing device are defined by an operating system of the computing device, and wherein the executable actions are executable by one or more applications installed on the computing device.

5. The method of claim 2, the method further comprising:

while the computing device is not in communication with the remote server, receiving, at the computing device, a second audio sample;

determining that the second audio sample matches one of the one or more stored audio samples;

identifying one or more words associated with the matching stored audio sample, the one or more words being stored in the local dictation database;

identifying a second executable action associated with the one or more words; and

executing, at the computing device, the second executable action.

6. The method of claim 2, wherein a plurality of executable actions are stored on the computing device, each executable action including an action text describing the executable action, the computing device having a plurality of applications, each application associated with one or more of the plurality of executable actions, wherein the computing device determines that the one or more words are associated with a first executable action of the computing device by:

identifying a group of executable actions that a currently open application of the computing device is capable of executing; and

determining that the action text of a first executable action matches the one or more words.

7. The method of claim 1, the method further comprising:

determining that the one or more words are not associated with an executable action of the computing device; and

displaying, in a user interface of a display at the computing device, an indication that the first audio sample was not recognized.

8. The method of claim 1, wherein the computing device is configured to determine that the first audio sample does not match any of the one or more stored audio samples by:

comparing the first audio sample with each of the one or more stored audio samples; and

determining, using a local speech recognition algorithm, that none of the one or more stored audio samples match the first audio sample within a margin of error.

9. The method of claim 1, the method further comprising:

initializing the local dictation database of the computing device.

10. The method of claim 9, wherein the computing device is configured to initialize the local dictation database by:

identifying, based on a text file stored at the computing device, one or more text strings associated with an application, the one or more text strings corresponding to commands recognizable by the application;

transmitting each of the one or more text strings to a remote server;

for each text string transmitted to the remote server, receiving an associated audio sample from the remote server, the audio sample being a vocalization of the associated text string; and

for each text string transmitted to the remote server, storing the associated audio sample and the text string in the local dictation database of the computing device.

11. The method of claim 10, wherein the text file includes one or more accessibility labels for providing accessibility services to the visually impaired.

12. The method of claim 1, the method further comprising:

determining that the one or more words is stored in the local dictation database in association with a second audio sample, such that the first audio sample and the second audio sample are associated with the one or more words.

13. The method of claim 1, wherein the first audio sample includes an audio recording of a voice command from the user.

14. A non-transitory computer-readable storage medium storing instructions executable by a computing device to perform a method for selectively interacting with a server to build a local dictation database for speech recognition at a device, the method comprising:

receiving, at a computing device associated with a user, a first audio sample;

determining, at the computing device, that the first audio sample does not match any of one or more audio samples stored in a local dictation database of the computing device;

transmitting the first audio sample to a remote server for detection of one or more words indicated by the first audio sample;

receiving data identifying the one or more words from the remote server; and

updating the local dictation database to store the first audio sample in association with the one or more words.

15. The non-transitory computer-readable storage medium of claim 14, the method further comprising:

determining that the one or more words are associated with a first executable action of the computing device.

16. The non-transitory computer-readable storage medium of claim 15, the method further comprising:

while the computing device is not in communication with the remote server, receiving, at the computing device, a second audio sample;

determining that the second audio sample matches one of the one or more stored audio samples;

identifying one or more words associated with the matching stored audio sample, the one or more words being stored in the local dictation database;

identifying a second executable action associated with the one or more words; and

executing, at the computing device, the second executable action.

17. The non-transitory computer-readable storage medium of claim 14, the method further comprising:

identifying, based on a text file stored at the computing device, one or more text strings associated with an application, the one or more text strings corresponding to commands recognizable by the application;

transmitting each of the one or more text strings to a remote server;

for each text string transmitted to the remote server, receiving an associated audio sample from the remote server, the audio sample being a vocalization of the associated text string; and

for each text string transmitted to the remote server, storing the associated audio sample and the text string in the local dictation database of the computing device.

18. One or more computing devices for selectively interacting with a server to build a local dictation database for speech recognition at a device, the one or more computing devices comprising:

one or more processors operable to execute one or more instructions to;

receive, at a computing device associated with a user, a first audio sample;

determine, at the computing device, that the first audio sample does not match any of one or more audio samples stored in a local dictation database of the computing device;

transmit the first audio sample to a remote server for detection of one or more words indicated by the first audio sample;

receive data identifying the one or more words from the remote server; and

update the local dictation database to store the first audio sample in association with the one or more words.

19. The one or more computing devices of claim 18, the one or more processors further operable to execute one or more instructions to:

identify, based on a text file stored at the computing device, one or more text strings associated with an application, the one or more text strings corresponding to commands recognizable by the application;

transmit each of the one or more text strings to a remote server;

for each text string transmitted to the remote server, receive an associated audio sample from the remote server, the audio sample being a vocalization of the associated text string; and

for each text string transmitted to the remote server, store the associated audio sample and the text string in the local dictation database of the computing device.

20. The one or more computing devices of claim 19, the one or more processors further operable to execute one or more instructions to:

while the computing device is not in communication with the remote server, receive, at the computing device, a second audio sample;

determine that the second audio sample matches one of the one or more stored audio samples;

identify one or more words associated with the matching stored audio sample, the one or more words being stored in the local dictation database;

identify a second executable action associated with the one or more words; and

execute, at the computing device, the second executable action.