Web server controls for web enabled recognition and/or audible prompting

• US 8,229,753 B2
• Filed: 10/21/2001
• Issued: 07/24/2012
• Est. Priority Date: 10/21/2001
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Galanes, Francisco M., Hon, Hsiao-Wuen, Jacoby, James D., Lecoueche, Renaud J., Potter, Stephen F., Warren, Susan M.
• Primary Examiner(s): Lerner, Martin
• Application Number:
  US10/046,131
• Publication Number:
  US 20030200080A1
• Time in Patent Office: 3,929 Days
• Field of Search: 704/270, 704/270.1, 704/275, 715/727, 715/760
• US Class Current: 704/270.1
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04M 1/72445 for supporting Internet browser applications
• CPC Class Codes: H04M 2250/74 with voice recognition means voice activated dialling H04M1/271; voice recognition algorithms G10L15/00
• CPC Class Codes: H04M 3/4938 comprising a voice browser which renders and interprets, e.g. VoiceXML

Abstract

Web server controls are provided for generating client side markups with recognition and/or audible prompting. Three approaches are disclosed for implementation of the controls.

Claims

1. A computer readable storage medium having instructions, which when implemented by a computer generate client side markup for a client in a server/client system, the instructions comprising:

a set of controls defined on an authoring page for a website for defining visual renderings and at least one of recognition and audible prompting on a client in a server/client system, each control having a first set of attributes defined on the authoring page related to visual rendering and a second set of attributes defined on the authoring page related to at least one of recognition and audibly prompting, wherein one of the second set of attributes for one of the controls relates to a grammar to use for recognition, the controls being related to client side markup executable by a client browser, the set of controls accessing and using a first library for the visual rendering and a second library for the at least one of recognition and audibly prompting, the first library including methods for generating the client side markup and a plurality of event handlers, the first library further comprising a plurality of website authoring tools, the second library including markup information for performing the at least one of recognition and audibly prompting; and

a module operable on a computer, the module being configured to receive the authoring page, and wherein the module is further configured to generate, using the first and second set of attributes provided from the controls on the authoring page, client side markup executable by the client browser on the client in the server/client system in accordance with the controls and the attributes of the controls to perform both visual rendering and at least one of recognition and audibly prompting.

2. The computer readable storage medium of claim 1 wherein said one of the second set of attributes provides a reference to a location of the grammar.

3. The computer readable storage medium of claim 1 wherein the grammar is for one of speech recognition, handwriting recognition, gesture recognition and visual recognition.

4. The computer readable storage medium of claim 3 wherein the controls relate to one of HTML, XHTML, cHTML, XML and WML.

5. The computer readable storage medium of claim 1 wherein the controls relate to one of HTML, XHTML, cHTML, XML and WML.

6. The computer readable storage medium of claim 1 wherein one of the second set of attributes for one of the controls provides instructions related to generating audible output.

7. The computer readable storage medium of claim 6 wherein the instructions comprise text and the attribute relates to converting the text to audible output.

8. The computer readable storage medium of claim 1 wherein one of the second set of attributes for one of the controls relates to a location of data for audible output.

9. The computer readable storage medium of claim 8 wherein the data comprises a prerecorded audio data file and the attribute relates to playing the prerecorded audio data file.

10. The computer readable storage medium of claim 8 wherein the data comprises text and the attribute relates to converting the text to audible output.

11. A computer readable storage medium having instructions defined on an authoring page for a website, which when implemented by a computer generate client side markup for a client in a server/client system, the instructions comprising:

a first set of visual controls defined on the authoring page having attributes defined on the authoring page related to a first modality of interaction with a user of the client that being visual renderings on the client device, the first set of controls being related to client side markup executable by a client browser; and

a second set of controls defined on the authoring page having attributes defined on the authoring page related to a second modality of interaction with a user of the client that being at least one of recognition and audibly prompting, wherein attributes related to audible prompting include at least one of inline text for text-to-speech conversion, location of data for audible rendering and playing of a prerecorded audio file, the second set of controls using at least one of the first controls, the second set of controls being related to client side markup executable by a client browser, the first set of controls accessing and using a first library for the visual renderings, the second set of controls accessing and using a second library for the at least one of recognition and audibly prompting, the first library including methods for generating the client side markup and a plurality of event handlers, the first library further comprising a plurality of website authoring tools, the second library including markup information for performing the at least one of recognition and audibly prompting; and

a module operable on a computer, the module being configured to receive the authoring page, which includes a plurality of the second set of controls, wherein the module is further configured to process the plurality of the second set of controls from the authoring page to generate client side markup from the first and second set of controls that is executable by the client browser on the client in the server/client system in accordance with the second set of controls and the attributes of the second set of controls for at least one of recognition and audibly prompting, and wherein the module is configured to use at least one of the first set of controls from the authoring page in order to generate markup herefrom when processing each of the second set of controls.

12. The computer readable storage medium of claim 11 wherein one of the attributes for the second set of controls relates to a grammar to use for recognition.

13. The computer readable storage medium of claim 12 wherein said one of the attributes provides a reference to a location of the grammar.

14. The computer readable storage medium of claim 12 wherein the grammar is for one of speech recognition, handwriting recognition, gesture recognition and visual recognition.

15. The computer readable storage medium of claim 14 wherein the first set of controls and the second set of controls relate to one of HTML, XHTML, cHTML, XML and WML.

16. The computer readable storage medium of claim 11 wherein the controls relate to one of HTML, XHTML, cHTML, XML and WML.

17. The computer readable storage medium of claim 11 wherein one of the second set of attributes for one of the controls provides instructions related to generating audible output.

18. The computer readable storage medium of claim 17 wherein the instructions comprise text and the attribute relates to converting the text to audible output.

19. The computer readable storage medium of claim 11 wherein one of the second set of attributes for one of the controls relates to a location of data for audible output.

20. The computer readable storage medium of claim 19 wherein the data comprises a prerecorded audio data file and the attribute relates to playing the prerecorded audio data file.

21. The computer readable storage medium of claim 19 wherein the data comprises text and the attribute relates to converting the text to audible output.

22. A computer readable storage medium having instructions, which when implemented by a computer generate client side markup for a client in a server/client system, the instructions comprising:

a first set of visual controls defined on an authoring page for a website having attributes defined on the authoring page related to a first modality of interaction with a user of the client that being visual renderings on the client device, the first set of controls being related to client side markup executable by a client browser; and

a second set of controls defined on the authoring page for defining desired operation on the client device having attributes defined on the authoring page related to a second modality of interaction with a user of the client that being at least one of recognition and audible prompting, wherein attributes related to recognition include at least one of location of grammar for use in recognition and confidence level thresholds for recognition and wherein attributes related to audible prompting include at least one of inline text for text-to-speech conversion, location of data for audible rendering and playing of a prerecorded audio file, the second set of controls being selectively associated with the first set of controls and wherein values of the second set of controls are synchronized with the first set of visual controls, and the second set of controls being related to client side markup executable a client browser, the first set of controls accessing and using a first library for the visual renderings, the second set of controls accessing and using a second library for the at least one of recognition and audibly prompting, the first library including methods for generating the client side markup and a plurality of event handlers, the first library further comprising a plurality of website authoring tools, the second library including markup information for performing the at least one of recognition and audibly prompting; and

a module operable on a computer, the module being configured to receive the authoring page, wherein the module is further configured to process the controls of the first set in the authoring page to generate client side markup for the first modality of interaction by incorporating the attributes in the controls that is executable by the client browser on the client in the server/client system in accordance with the controls of the first set and the attributes of the controls of the first set to perform both visual rendering, and wherein the module is further configured to process the controls of the second set to generate client side markup for the second modality of interaction by incorporating the attributes in the controls that is executable by the client browser on the client in the server/client system in accordance with the controls of the second set and the attributes of the controls of the second set in the authoring page to perform at least one of recognition and audibly prompting.

23. The computer readable storage medium of claim 22 wherein one of the attributes for the second set of controls relates to a grammar to use for recognition.

24. The computer readable storage medium of claim 23 wherein said one of the attributes provides a reference to a location of the grammar.

25. The computer readable storage medium of claim 23 wherein the grammar is for one of speech recognition, handwriting recognition, gesture recognition and visual recognition.

26. The computer readable storage medium of claim 25 wherein the first set of controls and the second set of controls relate to one of HTML, XHTML, cHTML, XML and WML.

27. The computer readable storage medium of claim 22 wherein the controls relate to one of HTML, XHTML, cHTML, XML and WML.

28. The computer readable storage medium of claim 22 wherein one of the second set of attributes for one of the controls provides instructions related to generating audible output.

29. The computer readable storage medium of claim 28 wherein the instructions comprise text and the attribute relates to converting the text to audible output.

30. The computer readable storage medium of claim 22 wherein one of the second set of attributes for one of the controls relates to a location of data for audible output.

31. The computer readable storage medium of claim 30 wherein the data comprises a prerecorded audio data file and the attribute relates to playing the prerecorded audio data file.

32. The computer readable storage medium of claim 30 wherein the data comprises text and the attribute relates to converting the text to audible output.

33. The computer readable storage medium of claim 22 wherein one of the attributes of the second set of controls relates to an identifier of the associated control of the first set of controls to form the association.

34. The computer readable storage medium of claim 22 wherein one of the attributes relates to whether the associated control of the second set is available for activation.

35. The computer readable storage medium of claim 34 wherein activation relates to generating markup.

36. The computer readable storage medium of claim 34 wherein activation relates to execution on the client.

37. The computer readable storage medium of claim 22 wherein the second set of controls activates another control of the second set.

38. The computer readable storage medium of claim 22 wherein the second set of controls comprise:

a question control for generating markup related to audible prompting of a question; and

an answer control for generating markup related to a grammar for recognition.

39. The computer readable storage medium of claim 38 wherein the question control activates the answer control.

40. The computer readable storage medium of claim 39 wherein the answer control includes a mechanism to associate a received result with one of the first set of controls.

41. The computer readable storage medium of claim 40 wherein mechanism includes binding the recognition value.

42. The computer readable storage medium of claim 41 wherein the mechanism includes issuing an event related to operation of binding.

43. The computer readable storage medium of claim 39 wherein the second set of controls comprise:

a command control for generating markup related to a grammar for one of navigation in the markup, help with a task, and repeating an audible prompt.

44. The computer readable storage medium of claim 39 wherein the second set of controls comprise:

a confirmation control for generating markup related to confirming that a recognized result is correct.

45. The computer readable storage medium of claim 44 wherein the confirmation control includes an attribute related to the recognized result to be confirmed.

46. The computer readable storage medium of claim 45 wherein the answer control includes an attribute related to a confidence level.

47. The computer readable storage medium of claim 45 wherein the confirmation control is activated as a function of a confidence level of a received result.

48. The computer readable storage medium of claim 47 wherein the confirmation control activates an accept control to accept the recognized result.

49. The computer readable storage medium of claim 47 wherein the confirmation control activates a deny control to deny the recognized result.

50. The computer readable storage medium of claim 47 wherein the confirmation control activates a correct control to correct the recognized result.

51. A computer implemented method for defining a website application on a server in a server/client architecture, the website application providing markup to a client for performing recognition and/or audible prompting on the client, the method comprising:

defining the website application by creating an authoring page with a first set of visual controls having attributes defined on the authoring page related to a first modality of interaction with a user of the client that being visual rendering on the client device, and with a second set of controls having attributes defined on the authoring page related to a second modality of interaction with the user of the client that being at least one of recognition and audibly prompting, wherein attributes related to audible prompting include at least one of inline text for text-to-speech conversion, location of data for audible rendering and playing of a prerecorded audio file, and wherein the first set of controls and the second set of controls being related to client side markup executable by a client browser, wherein defining includes selectively associating controls of the second set of controls with at least one control of the first set of visual controls, the first set of controls accessing and using a first library for the visual rendering, the second set of controls accessing and using a second library for the at least one of recognition and audibly prompting, the first library including methods for generating the client side markup and a plurality of event handlers, the first library further comprising a plurality of website authoring tools, the second library including markup information for performing the at least one of recognition and audibly prompting; and

processing the controls on the authoring page to generate client side markup including processing of the first set of visual controls to generate client side markup for the first modality of interaction executable by the client browser on the client in the server/client system in accordance with the controls of the first set and the attributes of the controls of the first set to perform visual rendering, and processing the controls of the second set to generate client side markup for the second modality of interaction by incorporating the attributes in the controls of the second set to perform at least one of recognition and audibly prompting.