Application abstraction with dialog purpose

• US 8,165,883 B2
• Filed: 04/28/2003
• Issued: 04/24/2012
• Est. Priority Date: 10/21/2001
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Galanes, Francisco M., Hon, Hsiao-Wuen, Jacoby, James D., Lecoeuche, Renaud J., Potter, Stephen F.
• Primary Examiner(s): Han, Qi
• Application Number:
  US10/426,053
• Publication Number:
  US 20040073431A1
• Time in Patent Office: 3,284 Days
• Field of Search: 704/270, 704/270.1, 704/231, 704/251, 704/275
• US Class Current: 704/270
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces

Abstract

Controls are provided for a web server to generate client side markups that include recognition and/or audible prompting. The controls comprise elements of a dialog such as a question, answer, confirmation, command or statement. A module forms a dialog by making use of the information carried in the controls.

Claims

1. A computer storage readable medium having instructions, which when executed on a computer generate client side markup for a client in a client/server system, the instructions comprising:

a first set of controls on an authoring page at a server located remote from a client, the first set of controls configured for use on the server located remote from the client for supporting audio rendering on the client and for defining a dialog for audibly interacting with a website and configured for dynamically generating client side markup in accordance with the dialog and with a determination of a type of the client, the first set of controls comprising at least a control configured at the server for generating markup at the server related to audible prompting of a question and for generating markup at the server related to a grammar recognizing audio input received at the client, each of the first set of controls having attributes configurable at the server, including a configurable attribute to indicate whether an associated control is activated, wherein the first set of controls are arranged in a hierarchical structure and repeatedly used with attributes having different values in order to define the dialog;

a second set of controls on the authoring page at the server configured for use on the server located remote from the client for supporting visual rendering on the client using a graphical user interface, wherein at least one of the first set of controls is associated with at least one of the second set of controls; and

a module, using the attributes provided directly from the first set of controls in the authoring page and when executed on the server, generates client side markup based on the dialog as a function of which controls are activated at the server, said markup including instructions to configure the client to maintain memory for a plurality of user inputs, wherein at least one user input is associated with input received through the graphical user interface as well as audio input recognized through speech recognition.

2. The computer storage readable medium of claim 1 wherein the set of controls includes means for defining a prompt generating markup for providing the question.

3. The computer storage readable medium of claim 1 wherein the module creates a dialog as a function of the first set of controls activated at the server, wherein each of the first set of controls contain built-in information identifying when each of the first set of controls are capable of being activated and wherein the dialog is based on a role of the first set of controls in the dialog activated at the server and the relationship of each of the first set of controls with other controls activated in the first set of controls.

4. The computer storage readable medium of claim 3 wherein the first set of controls includes a question control for generating markup related to providing the question and an answer control for generating markup related to the grammar for recognition.

5. The computer storage readable medium of claim 4 wherein the question control activates the answer control.

6. The computer storage readable medium of claim 5 wherein the first set of controls includes an attribute to indicate an order that each of the controls will be activated.

7. The computer storage readable medium of claim 1 wherein one of the first set of controls provides means for defining a command for generating markup related to a grammar for one of navigation in the markup, help with a task, and repeating an audible prompt.

8. The computer storage readable medium of claim 1 wherein one of the first set of controls provides means for defining a confirmation for generating markup related to confirming that a recognized result is correct.

9. The computer storage readable medium of claim 8 wherein the confirmation is activated as a function of a confidence level of a received result.

10. The computer storage readable medium of claim 9 wherein the control for generating markup related to a grammar for recognition includes an attribute related to a confidence level.

11. The computer storage readable medium of claim 8 wherein the first set of controls includes a confirmation control for generating markup related to confirm that a recognized result is correct.

12. The computer storage readable medium of claim 1 wherein the first set of controls includes means for defining a comparison to generate markup for comparing a recognized result with a selected value.

13. The computer storage readable medium of claim 1 wherein the first set of controls includes means for defining a comparison to generate markup for comparing a recognized result with another recognized result.

14. The computer storage readable medium of claim 1 and further comprising binding a recognition result to one of semantic items.

15. The computer storage readable medium of claim 14 wherein binding includes issuing an event related to operation of binding.

16. The computer storage readable medium of claim 1 and further comprising indicating that a semantic item has changed.

17. The computer storage readable medium of claim 1 and further comprising indicating that a semantic item has been confirmed.

18. The computer storage readable medium of claim 1 and further indicating when a semantic item is empty.

19. A computer storage readable medium having instructions, which when executed on a computer define a dialog, the instructions comprising:

a first set of controls on an authoring page at a server located remote from a client, the first set of controls configured for use on the server located remote from the client for supporting audio rendering on the client and for defining a dialog for audibly interacting with a website, and configured for dynamically generating client side markup in accordance with the dialog and with a determination of a type of the client, the controls comprising at least a question control having an attribute directly related to generating an audible prompt of a question, and an answer control having an attribute directly related to using a grammar for recognizing audio input received at the client, wherein the attribute directly related to using a grammar for recognizing audio input received at the client includes at least one of location of grammar for use in recognizing audio input received at the client and confidence level thresholds for recognizing audio input received at the client and wherein the attribute directly related to audible prompting includes at least one of inline text for text-to-speech conversion, location of data for audible rendering and playing of a prerecorded audio file, wherein the first set of controls are repeatedly used with attributes having different values in order to define the dialog, and wherein each of the controls further has having an attribute to indicate whether the associated control is available for activation and wherein at least some of the controls have a second attribute indicative of a hierarchical relationship of when controls are activated;

a second set of controls on the authoring page at the server configured for use on the server located remote from the client for supporting visual rendering on the client using a graphical user interface, wherein at least one of the first set of controls is associated with at least one of the controls of the second set of controls; and

a module, which when executed on the server and using the attributes provided directly from the first set of controls in the authoring page, generates markup by incorporating the attributes directly related to recognizing audio input received at the client and audible prompting in the markup that is sent to the remote client over a network, based on the dialog as a function of which of the first set of controls are activated, said markup including instructions to configure the client to maintain memory for a plurality of user inputs, wherein at least one user input is associated with input received through the graphical user interface as well as audio input recognized through speech recognition.

20. The computer storage readable medium of claim 19 wherein the first set of controls includes means for defining a prompt for providing the question.

21. The computer storage readable medium of claim 19 wherein the module creates a dialog as a function of the first set of controls activated at the server, wherein each of the first set of controls contain built-in information identifying when each of the first set of controls are capable of being activated and wherein the dialog is based on a role of the first set of controls in the dialog activated at the server and the relationship of each of the first set of controls with other controls activated in the first set of controls.

22. The computer storage readable medium of claim 21 wherein the first set of controls includes a question control related to providing the question and an answer control related to the grammar for recognition.

23. The computer storage readable medium of claim 22 wherein the question control activates the answer control.

24. The computer storage readable medium of claim 19 wherein the first set of controls includes an attribute to indicate an order that each of the controls will be activated.

25. The computer storage readable medium of claim 19 wherein one of the first set of controls provides means for defining a command related to a grammar for one of navigation in a website, help with a task, and repeating an audible prompt.

26. The computer storage readable medium of claim 19 wherein one of the first set of controls provides means for defining a confirmation control related to confirming that a recognized result is correct.

27. The computer storage readable medium of claim 26 wherein the confirmation is activated as a function of a confidence level of a received result.

28. The computer storage readable medium of claim 27 wherein the first set of controls includes means for defining a comparison to generate markup for comparing a recognized result with another recognized result.

29. The computer storage readable medium of claim 19 wherein the first set of control includes an attribute related to a confidence level.

30. The computer storage readable medium of claim 19 wherein the first set of controls includes means for defining a comparison to generate markup for comparing a recognized result with a selected value.

31. The computer storage readable medium of claim 19 wherein the markup includes instructions to ascertain that a user input has changed.

32. The computer storage readable medium of claim 19 wherein the markup includes instructions to ascertain that a user input has been confirmed.

33. The computer storage readable medium of claim 19 wherein the markup includes instructions to ascertain that a user input is empty.

34. A computer implemented method for performing recognition and/or audible prompting on a client in a client/server system, the method comprising:

defining a dialog with a first set of controls on an authoring page at a server located remote from a client, the set of controls configured for use on the server located remote from the client for supporting audio rendering on the client and comprising at least a question for generating an audible prompt of a question and a grammar recognizing audio input received at the client;

associating controls of a second set of controls on the authoring page at the server with the first set of controls, the second set of controls supporting visual rendering on the client using a graphical user interface; and

dynamically generating client side markup at the server in accordance with the defined dialog and with a determination of a type of the client on the server located remote from the client, said markup including instructions to configure the client to maintain memory for a plurality of user inputs, wherein at least one user input is associated with input received through the graphical user interface as well as audio input recognized through speech recognition wherein creating the dialog on the client includes executing the client side markup in a loop manner for the first set of controls activated at the server until a form having a plurality of values to be obtained from the user is complete.

35. The computer implemented method of claim 34 wherein defining the dialog includes means for defining a prompt for providing the question.

36. The computer implemented method of claim 34 wherein the first set of controls includes an attribute to indicate an order that each of the first set of controls will be activated.

37. The computer implemented method of claim 36 wherein creating the dialog further comprises creating the dialog as a function of the first set of controls activated at the server, wherein each of the first set of controls contain built-in information identifying when each of the first set of controls are capable of being activated and wherein the dialog is based on a role of the first set of controls in the dialog activated at the server and the relationship of each of the first set of controls with other controls activated in the first set of controls.

38. The computer implemented method of claim 37 wherein the first set of controls includes a question control related to providing the question and an answer control related to the grammar for recognition.

39. The computer implemented method of claim 38 wherein creating the dialog includes activation of the answer control through the question control.

40. The computer implemented method of claim 34 wherein creating the dialog includes activation of the confirmation as a function of a confidence level of a received result.