Distributed voice user interface

• US 7,769,591 B2
• Filed: 08/31/2006
• Issued: 08/03/2010
• Est. Priority Date: 04/12/1999
• Status: Expired due to Fees

• Current Assignee: Intellectual Ventures I LLC (Intellectual Ventures LLC)
• Original Assignee: Ben Frank Patent Holding LLC
• Inventors: White, George M., Shires, Glen E., Buteau, James J., Surace, Kevin J., Markman, Steven
• Primary Examiner(s): Lerner; Martin
• Application Number:
  US11/513,163
• Publication Number:
  US 20060293897A1
• Time in Patent Office: 1,433 Days
• Field of Search: 704/250, 704/251, 704/255, 704/258, 704/270, 704/270.1, 704/275, 434/185
• US Class Current: 704/270
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications

Abstract

A distributed voice user interface system includes a local device which receives speech input issued from a user. Such speech input may specify a command or a request by the user. The local device performs preliminary processing of the speech input and determines whether it is able to respond to the command or request by itself. If not, the local device initiates communication with a remote system for further processing of the speech input.

Claims

1. A local device comprising:

a primary functionality component;

an input component configured to receive speech input;

a processing component coupled to the input component, the processing component configured to;

identify keywords in the speech input,

determine whether the local device is capable of processing the speech input based on whether one or more keywords are identified in the speech input,

if the local device is capable of processing the speech input, process the speech input, generate corresponding local control signals, and transmit the local control signals to the primary functionality component to direct an action in the primary functionality component, and

if the local device is not capable of processing the speech input, extract feature parameters from the speech input for processing at a remote system, receive remote control signals from the remote system responsive to the remote system performing speech recognition on the feature parameters by storing an acoustic model of the feature parameters and recognizing a command based on a previously stored acoustic model associated with the local device to address specific characteristics of the feature parameters, and send the remote control signals to the primary functionality component; and

a transceiver coupled to the processing component and configured to establish communications between the local device and the remote system, wherein the communications comprise;

high bandwidth communications configured to return data supporting audio or video output at the local device, and

low bandwidth communications configured to return data supporting the remote control signals.

2. The local device of claim 1, further comprising a manual input component configured to allow manual initiation of the communications.

3. The local device of claim 1, wherein the processing component is configured to transmit the feature parameters to the remote system for speech recognition processing.

4. The local device of claim 1, further comprising a recording component configured to record the speech input.

5. The local device of claim 4, wherein the recording component is configured to play back the recorded speech input for transmission to the remote system.

6. The local device of claim 1, wherein the processing component comprises a speech generation engine configured to generate speech output.

7. The local device of claim 6, wherein the speech output generated by the speech generation engine is consistent with speech output generated by the remote system.

8. The local device of claim 1, wherein the local device is configured to analyze the speech input to extract feature parameters and to send a signal based upon the extracted feature parameters to the remote system.

9. The local device of claim 1, wherein both the local device and the remote system analyze the speech input in order to extract feature parameters.

10. The local device of claim 1, wherein the local device comprises at least one of a personal digital assistant, a smart telephone, a remote control device, a household appliance, an entertainment system, a security system, and a climate control system.

11. The local device of claim 1, wherein the processing component is further configured to replace a keyword in the set of known keywords based on the update from the remote system.

12. The local device of claim 1, wherein the processing component is further configured to add a keyword to the set of known keywords based on the update from the remote system.

13. A method of providing a voice interface for a primary functionality component in a local device, comprising:

receiving, at the local device, a speech input;

identifying keywords in the speech input;

establishing communications between the local device and a remote system, wherein the communications comprise;

high bandwidth communications configured to return data supporting audio or video output at the local device, and

low bandwidth communications configured to return data supporting the remote control signals;

determining, at the local device, whether the local device is capable of processing the speech input based on whether one or more keywords are identified in the speech input;

if the local device is capable of processing the speech input, then

processing the speech input at the local device,

generating corresponding local control signals, and

transmitting the local control signals to the primary functionality component to direct an action in the primary functionality component; and

if the local device is not capable of processing the speech input, then

extracting feature parameters from the speech input for processing at the remote system,

sending the feature parameters to the remote system for processing by storing an acoustic model of the feature parameters and recognizing a command based on a previously stored acoustic model associated with the local device to address specific characteristics of the feature parameters,

receiving remote control signals from the remote system responsive to the feature parameters via the low bandwidth communications, and

sending the remote control signals to the primary functionality component.

14. The method of claim 13, wherein establishing communications between the local device and the remote system comprises:

establishing the communications between the local device and the remote system upon manual initiation.

15. The method of claim 13, wherein establishing communications between the local device and a remote system comprises:

establishing the communications between the local device and the remote system upon identification of a wake up command.

16. The method of claim 13, further comprising:

transmitting the feature parameters to the remote system for processing.

17. The method of claim 13, further comprising:

recording the speech input.

18. The method of claim 17, further comprising:

playing back the recorded speech input for transmission to the remote system.

19. The method of claim 13, further comprising:

generating a first speech output.

20. The method of claim 19, further comprising:

receiving a second speech output from the remote system, wherein the first speech output is consistent with the second speech output.

21. The method of claim 13, further comprising:

analyzing the speech input for the feature parameters; and

sending a signal based upon the feature parameters to the remote system.

22. The method of claim 13, wherein modifying the set of known keywords based at least in part on the update from the remote system comprises replacing a keyword in the set of known keywords.

23. The method of claim 13, wherein modifying the set of known keywords based at least in part on the update from the remote system comprises adding a keyword to the set of known keywords.

24. The method of claim 13, further comprising:

storing a previous enunciation of a certain word.

25. A tangible computer readable medium having stored thereon computer-executable instructions that, if executed by a computing device, cause the computing device to perform a method comprising:

receiving, at the local device, a speech input;

identifying keywords in the speech input;

establishing communications between the local device and a remote system, wherein the communications comprise;

high bandwidth communications configured to return data supporting audio or video output at the local device, and

low bandwidth communications configured to return data supporting the remote control signals;

determining, at the local device, whether the local device is capable of processing the speech input based on whether one or more keywords are identified in the speech input;

if the local device is capable of processing the speech input, then

processing the speech input at the local device,

generating corresponding local control signals, and

transmitting the local control signals to the primary functionality component to direct an action in the primary functionality component; and

if the local device is not capable of processing the speech input, then

extracting feature parameters from the speech input for processing at the remote system,

sending the feature parameters to the remote system for processing by storing an acoustic model of the feature parameters and recognizing a command based on a previously stored acoustic model associated with the local device to address specific characteristics of the feature parameter,

receiving remote control signals from the remote system responsive to the feature parameters via the low bandwidth communications, and

sending the remote control signals to the primary functionality component.

26. The computer program product of claim 25, wherein establishing communications between the local device and the remote system comprises:

establishing the communications between the local device and the remote system upon manual initiation.

27. The computer program product of claim 25, wherein establishing communications between the local device and the remote system comprises:

establishing the communications between the local device and the remote system upon identification of a wake up command.

28. The computer program product of claim 25, wherein the method further comprises:

transmitting the feature parameters to the remote system for processing.

29. The computer program product of claim 25, wherein the method further comprises:

recording the speech input.

30. The computer program product of claim 29, wherein the method further comprises:

playing back the recorded speech input for transmission to the remote system.

31. The computer program product of claim 25, wherein the method further comprises:

generating a first speech output.

32. The computer program product of claim 31, wherein the method further comprises:

receiving a second speech output from the remote system, wherein the first speech output is consistent with the second speech output.

33. The computer program product of claim 25, wherein the method further comprises:

analyzing the speech input in order to extract feature parameters; and

sending a signal based upon the extracted feature parameters to the remote system.

storing a previous enunciation of a certain word.

34. The computer program product of claim 25, wherein modifying the set of known keywords based at least in part on the update from the remote system comprises replacing a keyword in the set of known keywords.

35. The computer program product of claim 25, wherein modifying the set of known keywords based at least in part on the update from the remote system comprises adding a keyword to the set of known keywords.

36. The computer program product of claim 25, wherein the method further comprises:

storing a previous enunciation of a certain word.

37. A local device comprising:

receiving means to receive a speech input;

identifying means to identify keywords in the speech input;

establishing means to establish communications between the local device and a remote system, wherein the communications comprise;

high bandwidth communications configured to return data supporting audio or video output at the local device, and

low bandwidth communications configured to return data supporting the remote control signals;

determining means to determine whether a local device is capable of processing the speech input based on whether one or more keywords are identified in the speech input;

first responding means to respond to the speech input if the determining means determines that the local device is capable of processing the speech input, comprising;

processing means to process the speech input at the local device,

generating means to generate corresponding local control signals, and

transmitting means to transmit the local control signals to a primary functionality component to direct an action in the primary functionality component; and

second responding means to respond to the speech input if the determining means determines that the local device is not capable of processing the speech input, comprising;

extracting means to extract feature parameters from the speech input for processing at the remote system,

first sending means to send the feature parameters to the remote system for processing by storing an acoustic model of the feature parameters and recognizing a command based on a previously stored acoustic model associated with the local device to address specific characteristics of the feature parameter,

receiving means to receive remote control signals from the remote system responsive to the feature parameters via the low bandwidth communications, and

second sending means to send the remote control signals to the primary functionality component.

38. A local device comprising:

a primary functionality component;

an input component configured to receive speech input;

a processing component coupled to the input component, the processing component configured to;

identify keywords in the speech input,

determine whether the local device is capable of processing the speech input based on whether one or more keywords are identified in the speech input,

if the local device is capable of processing the speech input, process the speech input, generate corresponding local control signals, and transmit the local control signals to the primary functionality component to direct an action in the primary functionality component, and

if the local device is not capable of processing the speech input, extract feature parameters from the speech input for processing at a remote system, receive remote control signals from the remote system responsive to the remote system performing speech recognition on the feature parameters by storing an acoustic model of the feature parameters and recognizing a command based on a previously stored acoustic model associated with the local device to address specific characteristics of the feature parameters, and send the remote control signals to the primary functionality component; and

a recording component configured to record the speech input and to play back the recorded speech input for transmission to the remote system.

39. A method of providing a voice interface for a primary functionality component in a local device, comprising:

receiving, at the local device, a speech input;

identifying keywords in the speech input;

determining, at the local device, whether the local device is capable of processing the speech input based on whether one or more keywords are identified in the speech input;

if the local device is capable of processing the speech input, then

processing the speech input at the local device,

generating corresponding local control signals, and

transmitting the local control signals to the primary functionality component to direct an action in the primary functionality component;

if the local device is not capable of processing the speech input, then

extracting feature parameters from the speech input for processing at a remote system,

sending the feature parameters to the remote system for processing by storing an acoustic model of the feature parameters and recognizing a command based on a previously stored acoustic model associated with the local device to address specific characteristics of the feature parameters,

receiving remote control signals from the remote system responsive to the feature parameters, and

sending the remote control signals to the primary functionality component;

recording the speech input; and

playing back the recorded speech input for transmission to the remote system.

40. A tangible computer readable medium having stored thereon computer-executable instructions that, if executed by a computing device, cause the computing device to perform a method comprising:

receiving, at the local device, a speech input;

identifying keywords in the speech input;

determining, at the local device, whether the local device is capable of processing the speech input based on whether one or more keywords are identified in the speech input;

if the local device is capable of processing the speech input, then

processing the speech input at the local device,

generating corresponding local control signals, and

transmitting the local control signals to the primary functionality component to direct an action in the primary functionality component;

if the local device is not capable of processing the speech input, then

extracting feature parameters from the speech input for processing at a remote system,

sending the feature parameters to the remote system for processing by storing an acoustic model of the feature parameters and recognizing a command based on a previously stored acoustic model associated with the local device to address specific characteristics of the feature parameter,

receiving remote control signals from the remote system responsive to the feature parameters, and

sending the remote control signals to the primary functionality component;

recording the speech input; and

playing back the recorded speech input for transmission to the remote system.

41. A local device comprising:

receiving means to receive a speech input;

identifying means to identify keywords in the speech input;

determining means to determine whether a local device is capable of processing the speech input based on whether one or more keywords are identified in the speech input;

first responding means to respond to the speech input if the determining means determines that the local device is capable of processing the speech input, comprising;

processing means to process the speech input at the local device,

generating means to generate corresponding local control signals, and

transmitting means to transmit the local control signals to a primary functionality component to direct an action in the primary functionality component;

second responding means to respond to the speech input if the determining means determines that the local device is not capable of processing the speech input, comprising;

extracting means to extract feature parameters from the speech input for processing at a remote system,

first sending means to send the feature parameters to the remote system for processing by storing an acoustic model of the feature parameters and recognizing a command based on a previously stored acoustic model associated with the local device to address specific characteristics of the feature parameter,

receiving means to receive remote control signals from the remote system responsive to the feature parameters, and

second sending means to send the remote control signals to the primary functionality component;

recording means to record the speech input; and

playback means to play back the recorded speech input for transmission to the remote system.