Assisted multi-modal dialogue

• US 20050091059A1
• Filed: 08/29/2003
• Published: 04/28/2005
• Est. Priority Date: 08/29/2003
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Lecoeuche, Renaud J.
• Granted Patent: US 8,311,835 B2
• Time in Patent Office: Days
• US Class Current: 704/270.1
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04M 1/72445 for supporting Internet browser applications
• CPC Class Codes: H04M 2250/74 with voice recognition means voice activated dialling H04M1/271; voice recognition algorithms G10L15/00

Abstract

Controls are provided for a web server to generate client side markups that include recognition and/or audible prompting. The controls are organized in collections to obtain information pertaining to different topics. Each collection of controls create a separate dialog. In this manner, the collections can be selectively specified to execute the corresponding dialog.

Claims

1. A computer readable medium having instructions, which when executed on a computer generate client side markup for a client in a client/server system, the instructions comprising:

a set of controls for defining a dialog, the controls comprising at least a control for generating markup related to audible prompting of a question and for generating markup related to a grammar for recognition, the controls being organized in a set of collections based on topics of information to be obtained; and

a module, when executed on the client, that creates a dialog that starts and stops as a function of the controls in at least one of the set of collections.

2. The computer readable medium of claim 1 wherein a collection is defined as comprising a plurality of other collections.

3. The computer readable medium of claim 2 wherein each collection includes an identifier, and wherein the module the dialog is created as a function of identifying an identifier.

4. The computer readable medium of claim 1 and further comprising a second manager module to identify a collection for the first-mentioned module.

5. The computer readable medium of claim 1 wherein the module is adapted to execute the dialog in a voice-only scenario and a multi-modal scenario.

6. The computer readable medium of claim 5 wherein the module is adapted to render prompts to the user as a function of the dialog being executed in the voice-only scenario and the multi-modal scenario.

7. The computer readable medium of claim 6 wherein the module is adapted to await user responses differently as a function of rendering prompts in the voice-only scenario and the multi-modal scenario.

8. The computer readable medium of claim 6 wherein the module is adapted to suspend execution of the dialog after visually rendering a prompt and resume execution of the dialog upon receiving an input associated with activation of a visual prompt.

9. The computer readable medium of claim 5 wherein the module is adapted to selectively switch execution of the dialog in the voice-only scenario and the multi-modal scenario.

10. The computer readable medium of claim 9 wherein the module is adapted to execute the dialog in the voice-only scenario and the multi-modal scenario as a function of a default.

11. The computer readable medium of claim 1 wherein each collection includes an identifier, and wherein the dialog is created as a function of ascertaining an identifier.

12. The computer readable medium of claim 11 and further comprising instructions for identifying a textbox selected for input by a user and instructions for ascertaining the collection associated with the textbox.

13. The computer readable medium of claim 12 wherein the module is adapted to stop the dialog as a function of the user selecting another textbox.

14. The computer readable medium of claim 12 wherein the module is adapted to stop the dialog as a function of the user selecting a stop button.

15. The computer readable medium of claim 12 wherein the module is adapted to stop the dialog as a function of the user audibly providing a stop command.

16. The computer readable medium of claim 12 wherein the set of controls includes a confirmation control to confirm that a recognized result of input speech is correct, and wherein the module is adapted to stop the dialog as a function of the user not responding to a rendering of the recognized result provided by the confirmation control.

17. The computer readable medium of claim 1 and further comprising:

a set of visual controls having attributes for visual rendering on the client device, wherein the first-mentioned set of controls are selectively associated with the first set of controls.

18. The computer readable medium of claim 17 wherein the first-mentioned set of controls includes means for maintaining data apart from the associated control of the set of visual controls, said means for maintaining associating the first-mentioned set of controls with the control of the set of visual controls.

19. The computer readable medium of claim 18 wherein the module is adapted to modify data in said means for maintaining whether said data is provided in the voice-only scenario and the multi-modal scenario.

20. The computer readable medium of claim 19 wherein the means for maintaining data includes maintaining a set of items for corresponding data, and wherein at least some of the items are individually associated with controls of the set of visual controls, and wherein states are maintained for at least some of the items, the states including if the item is empty and if the item has been confirmed.

21. A computer implemented method for performing recognition and/or audible prompting on a computer, the method comprising:

defining a plurality of dialogs with a set of controls, the set of controls comprising at least a question for generating an audible prompt of a question and a grammar used for recognition, the controls being organized in a set of collections based on topics of information to be obtained; and

creating a complete dialog by specifying collections sequentially and executing the corresponding dialog that starts and stops as a function of the controls in each of the specified collections.

22. The method of claim 21 wherein creating a complete dialog by specifying collections sequentially comprises defining another collection as comprising a plurality of the collections.

23. The method of claim 21 wherein each collection includes an identifier, and wherein creating a complete dialog comprises specifying identifiers of the collections.

24. The method of claim 21 wherein each of the dialogs is adapted for execution in a voice-only scenario and a multi-modal scenario.

25. The method of claim 21 wherein defining the dialogs includes a second set of visual controls having attributes for visual rendering on the client device, and wherein the method further comprises selectively associating controls of the first-mentioned set of controls with at least one control of the set of visual controls.

26. The computer implemented method of claim 25 wherein the first-mentioned set of controls includes means for maintaining a set of items for corresponding recognized results apart from the associated controls of the set of visual controls;

and wherein selectively associating includes associating at least some of the items with controls of the first set of controls.