Scalable low resource dialog manager

• US 6,513,009 B1
• Filed: 12/14/1999
• Issued: 01/28/2003
• Est. Priority Date: 12/14/1999
• Status: Expired due to Term

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Fernhout, Paul Derek, Frank, David Carl, Comerford, Liam David
• Primary Examiner(s): Dorvil, Richemond
• Assistant Examiner(s): Opsasnick, Michael N.
• Application Number:
  US09/460,961
• Time in Patent Office: 1,141 Days
• Field of Search: 704/270.1, 704/270
• US Class Current: 704/270
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/228 of application context

Abstract

A spoken language interface between a user and at least one application or system includes a dialog manager operatively coupled to the application or system, an audio input system, an audio output system, a speech decoding engine and a speech synthesizing engine; and at least one user interface data set operatively coupled to the dialog manager, the user interface data set representing spoken language interface elements and data recognizable by the application. The dialog manager enables connection between the input audio system and the speech decoding engine such that a spoken utterance provided by the user is provided from the input audio system to the speech decoding engine. The speech decoding engine decodes the spoken utterance to generate a decoded output which is returned to the dialog manager. The dialog manager uses the decoded output to search the user interface data set for a corresponding spoken language interface element and data which is returned to the dialog manager when found, and provides the spoken language interface element associated data to the application for processing in accordance therewith. The application, on processing that element, provides a reference to an interface element to be spoken. The dialog manager enables connection between the audio output system and the speech synthesizing engine such that the speech synthesizing engine which, accepting data from that element, generates a synthesized output that expresses that element, the audio output system audibly presenting the synthesized output to the user.

Claims

1. Apparatus for providing a spoken language interface between a user and at least one application or system, the apparatus operating in accordance with a computer processing system including a processor, an audio input system for receiving speech data provided by the user, an audio output system for outputting speech data to the user, a speech decoding engine and a speech synthesizing engine, the apparatus comprising:

a dialog manager operatively coupled to the application or system, the audio input system, the audio output system, the speech decoding engine and the speech synthesizing engine; and

at least one user interface data set operatively coupled to the dialog manager, the user interface data set representing spoken language interface elements and data recognizable by the application;

wherein;

(i) the dialog manager enables connection between the input audio system and the speech decoding engine such that the spoken utterance provided by the user is provided from the input audio system to the speech decoding engine;

(ii) the speech decoding engine decodes the spoken utterance to generate a decoded output which is returned to the dialog manager;

(iii) the dialog manager uses the decoded output to search the user interface data set for a corresponding spoken language interface element and data which is returned to the dialog manager when found;

(iv) the dialog manager provides the spoken language interface element associated data to the application or system for processing in accordance therewith;

(v) the application, on processing that element, provides a reference to an interface element to be spoken;

(vi) the dialog manager enables connection between the audio output system and the speech synthesizing engine such that the speech synthesizing engine which, accepting data from that element, generates a synthesized output that expresses that element; and

(vii) the audio output system audibly presenting the synthesized output to the user.

2. The apparatus of claim 1, further wherein:

(i) the speech decoding engine is unable to decode the spoken utterance and returns error information to the dialog manager;

(ii) the dialog manager uses the error information to search the user interface data set for a corresponding spoken language interface element which is returned to the dialog manager when found;

(iii) the dialog manager enables connection between the audio output system and the speech synthesizing engine such that the speech synthesizing engine which, accepting data from that element, generates a synthesized output that expresses that element; and

(iv) the audio output system audibly presenting the synthesized output to the user.

3. The apparatus of claim 1, wherein, at the start of execution, the dialog manager performs a default initialization of one or more parameters of the speech decoding engine in accord with one or more settings provided in a hardware profile data set.

4. The apparatus of claim 1, wherein the dialog manager is responsive to one or more additional engines associated with the computer processing system for providing speech related functions, the one or more engines also being operatively coupled to the dialog manager.

5. The apparatus of claim 4, wherein the one or more engines include at least one of a user verification engine, an audio recording engine and an audio playback engine.

6. The apparatus of claim 1, wherein the dialog manager addresses a foreground application.

7. The apparatus of claim 1, wherein the dialog manager addresses a background application.

8. The apparatus of claim 1, wherein progress of one or more operations of the dialog manager is reported to the application or system to inform the application or system that a requested service is complete and additional services may be requested.

9. The apparatus of claim 1, wherein the dialog manager comprises an engine manager component, the engine manager component being directly coupled to the speech decoding engine and the speech synthesizing engine and providing a messaging interface between the engines and the dialog manager.

10. The apparatus of claim 1, wherein the dialog manager comprises an interpreter component, the interpreter component being directly coupled to the user interface data set and providing a searching mechanism for the dialog manager.

11. The apparatus of claim 1, wherein the dialog manager causes a new user interface data set structure to be allocated and loaded with data in response to a request by an application or system.

12. The apparatus of claim 1, wherein the dialog manager causes a new element to be added to an existing user interface data set structure.

13. The apparatus of claim 1, wherein the user interface data set includes at least one of a vocabulary data file, a prompt data file, a script data file, and a profile data file.

14. The apparatus of claim 1, wherein the computer processing system is a personal computer.

15. The apparatus of claim 1, wherein the computer processing system is portable by the user.

16. The apparatus of claim 1, wherein the dialog manager is operative to maintain a history of use associated with an application.

17. The apparatus of claim 1, wherein the dialog manager is operative to maintain a history of use associated with a system feature.

18. The apparatus of claim 1, wherein the dialog manager is operative to maintain a history of use where an accumulated number of uses is recorded.

19. The apparatus of claim 1, wherein the dialog manager is operative to maintain a history of use where an accumulated number of uses is incremented until it reaches a predetermined value.

20. The apparatus of claim 1, wherein the dialog manager is operative to maintain a history of use where a time of last use is recorded.

21. The apparatus of claim 1, wherein the dialog manager is operative to select among prompts to find prompts of reducing complexity corresponding to the increasing accumulated experience of the user.

22. The apparatus of claim 1, wherein the dialog manager is operative to select among prompts to find prompts of complexity corresponding to a measure of the time since user'"'"'s last experience with a system feature.

23. The apparatus of claim 1, wherein the dialog manager is operative to replace escape characters in prompt text with values provided in a prompt request message.

24. The apparatus of claim 1, wherein the dialog manager is operative to replace escape characters in prompt text with values provided in messages preceding a prompt request message.

25. The apparatus of claim 1, wherein the user interface data set includes at least one file providing default properties for the behavior of a physical interface.

26. The apparatus of claim 1, wherein user interface data set includes profiles associated with individual applications such that application presentation is customized.

27. The apparatus of claim 1, wherein the dialog manager forms a hierarchy of spoken addresses descending from the application, through the dialog manager, to the engines.

28. The apparatus of claim 1, further including means for searching a built-in command set of the dialog manager.

29. The apparatus of claim 28, further including means for searching the user interface data set in order from application to the operating system of the computer processing system to the dialog manager.

30. The apparatus of claim 1, wherein the dialog manager is responsive to changing the properties thereof by receiving commands described in the user interface data set.

31. The apparatus of claim 1, wherein the dialog manager is scaled by adding engines.

32. The apparatus of claim 1, wherein the dialog manager is scaled by adding scripts.

33. The apparatus of claim 1, wherein the dialog manager is scaled by adding vocabulary and prompt files.

34. The apparatus of claim 1, wherein the dialog manager is scaled by removing components.

35. The apparatus of claim 1, wherein the user interface data set is changed to match the active application.

36. The apparatus of claim 1, wherein a power state of components of the computer processing system is responsive to a command reflecting the activity of the dialog.

37. The apparatus of claim 1, wherein the user interface data set comprises synonyms in vocabulary files.

38. The apparatus of claim 1, wherein the user interface data set comprises prompts composed from strings and variables passed in a prompt service request message.

39. The apparatus of claim 1, wherein the user interface data set comprises prompts composed from strings and variables passed in messages preceding a prompt service request message.

40. The apparatus of claim 1, wherein the user interface data set comprises prompts composed from strings and prerecorded data.

41. The apparatus of claim 1, wherein progress of one or more operations of the dialog manager is reported to the application or system to inform the application or system that a requested service is complete.

42. The apparatus of claim 1, further including means for searching the at least one user interface data set in a predetermined order.

43. A method for providing a spoken language interface between a user and at least one application or system, the method comprising the steps of:

enabling connection between an input audio system and a speech decoding engine such that a spoken utterance provided by the user is provided from the input audio system to the speech decoding engine;

decoding the spoken utterance to generate a decoded output which is returned to a dialog manager;

using the decoded output to search a user interface data set for a corresponding spoken language interface element and data which is returned to the dialog manager when found;

providing the spoken language interface element associated data to the application or system for processing in accordance therewith;

the application, on processing the spoken language interface element, providing a reference to an interface element to be spoken;

enabling connection between an audio output system and a speech synthesizing engine such that the speech synthesizing engine which, accepting data from that element, generates a synthesized output that expresses that element; and

audibly presenting the synthesized output to the user.

44. An article of manufacture for providing a spoken language interface between a user and at least one application or system, comprising a machine readable medium containing one or more programs which when executed implement the steps of:

enabling connection between an input audio system and a speech decoding engine such that a spoken utterance provided by the user is provided from the input audio system to the speech decoding engine;

decoding the spoken utterance to generate a decoded output which is returned to a dialog manager;

using the decoded output to search a user interface data set for a corresponding spoken language interface element and data which is returned to the dialog manager when found;

providing the spoken language interface element associated data to the application or system for processing in accordance therewith;

the application, on processing the spoken language interface element, providing a reference to an interface element to be spoken;

enabling connection between an audio output system and a speech synthesizing engine such that the speech synthesizing engine which, accepting data from that element, generates a synthesized output that expresses that element; and

audibly presenting the synthesized output to the user.