Voice controlled wireless communication device system

• US 8,315,878 B1
• Filed: 03/18/2011
• Issued: 11/20/2012
• Est. Priority Date: 08/09/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Burns, Stephen S., Kowitz, Mickey W.
• Primary Examiner(s): SKED, MATTHEW J
• Application Number:
  US13/051,167
• Time in Patent Office: 613 Days
• Field of Search: None
• US Class Current: 704/275
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: H04M 2250/74 with voice recognition means
• CPC Class Codes: H04M 3/4936 Speech interaction details speech recognition per se G10L15/00

Abstract

A wireless communication device is disclosed that accepts recorded audio data from an end-user. The audio data can be in the form of a command requesting user action. Likewise, the audio data can be converted into a text file. The audio data is reduced to a digital file in a format that is supported by the device hardware, such as a .wav, .mp3, .vnf file, or the like. The digital file is sent via secured or unsecured wireless communication to one or more server computers for further processing. In accordance with an important aspect of the invention, the system evaluates the confidence level of the of the speech recognition process. If the confidence level is high, the system automatically builds the application command or creates the text file for transmission to the communication device. Alternatively, if the confidence of the speech recognition is low, the recorded audio data file is routed to a human transcriber employed by the telecommunications service, who manually reviews the digital voice file and builds the application command or text file. Once the application command is created, it is transmitted to the communication device. As a result of the present invention, speech recognition in the context of communications devices has been shown to be accurate over 90% of the time.

Claims

1. A computer-implemented method comprising:

receiving a contact list from a wireless communication device;

receiving, at a first server computer, a voice command recorded via the wireless communication device;

initiating a speech recognition process that identifies the voice command, the speech recognition process using the contact list to assist in automatic translation of the voice command;

in response to determining a type of voice command, transferring the voice command to a second server computer that processes the voice command;

constructing an application command based on the voice command; and

transmitting the application command to the wireless communication device, the application command including instructions that cause a corresponding application on the wireless communication device to execute the application command.

2. The computer-implemented method of claim 1, further comprising providing an interface that presents the voice command along with the contact list for manual review and identification of the voice command.

3. The computer-implemented method of claim 1, wherein the first server computer manages contact names by resolving names, identified in a specific voice command without a spoken middle initial, to a corresponding name, from the contact list, that includes a middle initial.

4. The computer-implemented method of claim 1, wherein determining the type of voice command include identifying that the type of voice command includes confidential information, and, in response, the transferring the voice command to an additional server computer designated for receiving confidential information.

5. The computer-implemented method of claim 1, wherein initiating the speech recognition process includes assigning a confidence level to an identification of the voice command, the confidence level indicating how confident the system is of accurate recognition of the voice command;

wherein in response to an assigned confidence level determined to be at or above a predefined level, automatically constructing the application command;

wherein in response to the assigned confidence level determined to be below the predefined level, routing the voice command to an interface for manual review and transcription of the voice command;

wherein transmitting the application command to the wireless communication device includes transmitting a manually-constructed command to the wireless communication device.

6. The computer-implemented method of claim 5, further comprising:

training the speech recognition process to better identify voice commands in response to received manual input;

routing to a manual interface based on transcriber criteria, wherein the transcriber criteria includes a type of voice command, a workload of a particular transcriber, and a measure of typing ability of the particular transcriber; and

in response to routing the voice command to the manual interface, automatically displaying historical data associated with the user to expedite manual processing of the voice command.

7. The system of claim 6, further comprising:

including at least one additional server computer comprising at least one speech recognition process;

wherein the voice command is directed to the additional server computer based on user-based server criteria; and

wherein at least one of the user-based server criteria includes server activity.

8. The computer-implemented method of claim 1, wherein the voice command includes a unique device identifier that identifies the wireless communication device, the unique device identifier being used to track system usage.

9. The system of claim 1, wherein the voice command is directed to the additional server computer based on user-based server criteria including server activity.

10. A system for controlling wireless communication devices via voice commands, the system comprising:

a processor; and

a memory coupled to the processor, the memory storing instructions that, when executed by the processor, cause the system to perform the operations of;

receiving a contact list from a wireless communication device;

receiving, at a first server, a voice command recorded via the wireless communication device;

initiating a speech recognition process that identifies the voice command, the speech recognition process using the contact list to assist in automatic translation of the voice command;

in response to determining a type of voice command, transferring the voice command to a second server that processes the voice command;

constructing an application command based on the voice command; and

transmitting the application command to the wireless communication device, the application command including instructions that cause a corresponding application on the wireless communication device to execute the application command.

11. A computer-implemented method for controlling wireless communication devices via voice commands, the computer-implemented method comprising:

transmitting a contact list from a wireless communication device to a server computer;

initiating a software-based application resident on the wireless communication device that prompts for a voice command as input;

recording a user voice command via the wireless communication device and communicating the voice command to the server computer;

transmitting, to a first server computer, a voice command recorded via the wireless communication device;

receiving an application command at the wireless communication device from the first server computer, the application command having been constructed based on using a speech recognition process that identifies the voice command using the contact list to assist in automatic translation of the voice command, determines a type of voice command and transfers the voice command to a second server computer that processes the voice command; and

executing the application command on the wireless communication device.

12. A computer-implemented method of controlling wireless communication devices via voice commands, the computer-implemented method comprising:

initiating a software-based application resident on a wireless communication device that prompts input of a voice command;

recording a user voice command via the wireless communication device and communicating the voice command to a server computer;

receiving an application command at the wireless communication device from the server computer, the application command having been constructed based on using a speech recognition process that assigns a confidence level to an identification of the voice command to determine accuracy of the speech recognition process such that in response to an assigned confidence level being below a predetermined level, the application command is created by using manual transcription input, the voice command also having received additional processing via at least one additional server that executes a content-oriented speech recognition process that uses a particular lexicon; and

executing the application command on the wireless communication device.

13. A computer-implemented method for voice-controlled systems, the computer-implemented method comprising:

receiving, from a wireless communication device, a voice command recorded via the wireless communication device;

initiating a speech recognition process, at a first server computer, that identifies the voice command;

determining, based on results of the speech recognition process of the first server computer, that the voice command requires additional processing;

transmitting the voice command to a second server computer;

executing, at the second server computer, a content-oriented speech recognition process that uses a particular lexicon;

constructing an application command based on the voice command; and

transmitting the application command to the wireless communication device, the application command including instructions that cause a corresponding application on the wireless communication device to execute the application command.

14. A computer-implemented method for voice-controlled systems, the computer-implemented method comprising:

receiving, from a wireless communication device, a voice command recorded via the wireless communication device;

initiating multiple speech recognition process, at a first server computer, that generate multiple identifications of the voice command;

in response to completing the multiple speech recognition processes, evaluating confidence levels assigned to each identification of the voice command, the confidence levels indicating a measure of accurate recognition of the voice command for each identification;

in response to determining an identification of the voice command with a highest confidence level, automatically creating an application command based on the identification with the highest confidence level; and

in response to determining that all assigned confidence levels are below a critical level of confidence, routing the voice command to an interface for manual review and transcription of the voice command;

constructing an application command based on the voice command; and

transmitting the application command to the wireless communication device, the application command including instructions that cause a corresponding application on the wireless communication device to execute the application command.

15. The computer-implemented method of claim 14, wherein the voice command includes instructions to receive multimedia content.

16. A computer-implemented method for voice-controlled systems, the computer-implemented method comprising:

receiving, from a wireless communication device, a voice command recorded via the wireless communication device;

initiating a speech recognition process, at a first server computer, that identifies the voice command;

evaluating a confidence level of the speech recognition process associated with the identified voice command to determine accuracy of the speech recognition process;

in response to the confidence level being at or above a predefined level, constructing an application command based on the voice command;

in response to the confidence level being below the predefined level, routing the voice command for manual transcription and manual creation of the application command, wherein routing the voice command for manual transcription is based on transcriber criteria, the transcriber criteria including a type of voice command;

constructing an application command based on the voice command; and

transmitting the application command to the wireless communication device, the application command including instructions that cause a corresponding application on the wireless communication device to execute the application command.

17. The computer-implemented method of claim 16 further comprising:

wherein in response to routing the voice command for manual transcription, displaying historical data associated with a user that corresponds to the voice command; and

in response to receiving input from manual processing, displaying a prompt to manually update a speech recognition grammar file associated with the user.

18. The computer-implemented method of claim 17, further comprising training the speech recognition process to better understand voice commands, the training including manual training of the speech recognition process in response to a given confidence level meeting a predetermined threshold.

19. The computer-implemented method of claim 16, wherein at least one of the transcriber criteria includes a workload associated with a particular transcriber, a measure of typing ability associated with the particular transcriber, or past experience of the particular transcriber in relation to a user associated with the voice command.

20. The computer-implemented method of claim 16, further comprising:

routing the voice command to a second server computer in response to the first server computer identifying that the voice command requires additional processing, the second server computer including a content-oriented speech recognition process and that uses a particular lexicon.