Markup language extensions for web enabled recognition

• US 20020169806A1
• Filed: 04/05/2002
• Published: 11/14/2002
• Est. Priority Date: 05/04/2001
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wang, Kuansan, Hon, Hsiao-Wuen
• Granted Patent: US 7,610,547 B2
• Time in Patent Office: Days
• US Class Current: 715/541
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/117 Tagging; Marking up details of markup languages G06F40/143; Designating a block; Setting of attributes style sheets, e.g. eXtensible Stylesheet Language Transformation [XSLT], G06F40/154
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: H04M 1/271 controlled by voice recognition
• CPC Class Codes: H04M 1/72445 for supporting Internet browser applications
• CPC Class Codes: H04M 2207/40 terminals with audio html browser
• CPC Class Codes: H04M 2250/74 with voice recognition means voice activated dialling H04M1/271; voice recognition algorithms G10L15/00
• CPC Class Codes: H04M 3/493 Interactive information services, e.g. directory enquiries ; Arrangements therefor, e.g. interactive voice response [IVR] systems or voice portals
• CPC Class Codes: H04M 3/4936 Speech interaction details speech recognition per se G10L15/00

Abstract

A markup language for execution on a client device in a client/server system includes extensions for recognition.

Claims

1. A computer readable medium having a markup language for execution on a client device in a client/server system, the markup language comprising an instruction indicating a grammar to associate with input entered through the client device.

2. The computer readable medium of claim 1 wherein the instruction indicating a grammar provides a reference to a location of the grammar.

3. The computer readable medium of claim 1 wherein the grammar is for one of speech recognition, handwriting recognition, gesture recognition and visual recognition.

4. The computer readable medium of claim 3 wherein the instruction indicating a grammar includes a reference to a grammar for speech recognition.

5. The computer readable medium of claim 4 wherein the markup language comprises one of HTML, XHTML, cHTML, XML, and WML.

6. The computer readable medium of claim 4 wherein the markup language comprises a scripting language.

7. The computer readable medium of claim 4 wherein the markup language comprises synchronized multimedia markup language.

8. The computer readable medium of claim 1 wherein the markup language includes an instruction for receiving a recognition result pertaining to recognized speech and associating the result with a data field on the client device.

9. The computer readable medium of claim 8 wherein the instruction for receiving a recognition result associates the recognition result to a plurality of data fields.

10. The computer readable medium of claim 1 wherein the markup language includes an instruction to audibly render an indication when the speech was not recognized.

11. The computer readable medium of claim 1 wherein the markup language includes an instruction to visually render an indication when the speech was not recognized.

12. The computer readable medium of claim 1 wherein the markup language includes an instruction to set a time period related to speech recognition.

13. The computer readable medium of claim 12 wherein the time period indicates a maximum period of silence from a start recognition instruction.

14. The computer readable medium of claim 12 wherein the time period indicates a maximum period before a result must be returned after detection of speech.

15. The computer readable medium of claim 12 wherein the time period indicates a maximum period before a result must be returned after a start recognition instruction.

16. The computer readable medium of claim 12 wherein the time period indicates a minimum period of silence to indicate an end of speech.

17. The computer readable medium of claim 1 wherein the markup language includes an instruction indicating a confidence measure to use for speech recognition.

18. The computer readable medium of claim 1 wherein the markup language includes an instruction for taking action based on a value previously received.

19. The computer readable medium of claim 1 wherein the grammar comprises a DTMF grammar.

20. The computer readable medium of claim 19 wherein the markup language includes an instruction for handling a DTMF event.

21. The computer readable medium of claim 1 wherein the markup language includes an instruction for handling a call control event.

22. The computer readable medium of claim 1 wherein the markup language includes an instruction indicating for audibly prompting.

23. The computer readable medium of claim 22 wherein the markup language includes an instruction for prompting by converting text to speech.

24. The computer readable medium of claim 22 wherein the markup language includes an instruction for prompting by playing a selected audio stream.

25. The computer readable medium of claim 22 wherein the markup language includes an instruction for retrieving a value for use in prompting.

26. The computer readable medium of claim 22 wherein the markup language includes an instruction for related to halting prompting.

27. The computer readable medium of claim 1 wherein the input comprises a text string.

28. The computer readable medium of claim 27 wherein the markup language includes an instruction for receiving the text string from another component.

29. The computer readable medium of claim 28 wherein markup language associates the grammar to process the text string.

30. A computer readable medium including instructions readable by a computer which, when implemented, cause the computer to handle information by performing steps comprising:

receiving a markup language page from a web server having a field for input data;

receiving input from a user related to the field; and

sending data indicative of the input and an indication of a grammar for recognition.

31. The computer readable medium of claim 30 wherein the indication provides a reference to locations of the grammars.

32. The computer readable medium of claim 30 wherein the indication includes a reference to a language for speech recognition.

33. The computer readable medium of claim 30 wherein the markup language comprises one of HTML, XHTML, cHTML, XML and WML.

34. The computer readable medium of claim 30 wherein the markup language comprises a scripting language.

35. The computer readable medium of claim 30 wherein the markup language comprises a synchronized markup language.

36. The computer readable medium of claim 30 and wherein the input comprises speech, and further including instructions, which when implemented, comprise:

processing the input speech to provide data indicative of the input speech.

37. The computer readable medium of claim 36 wherein processing includes normalizing the data indicative of the input speech.

38. The computer readable medium of claim 36 wherein processing includes generating data indicative of parameters of the input speech.

39. The computer readable medium of claim 30 wherein the input comprises speech and the grammar comprises a speech recognition grammar.

40. The computer readable medium of claim 30 wherein the input comprises DTMF input and the grammar comprises a DTMF grammar.

41. A method for recognition in a client/server network, the method comprising:

receiving a markup language page having a field for input data on a client device connected to a network from a web server at an address on the network different than the client device;

receiving input from a user related to the field on the client device; and

sending data indicative of the input and an indication of a grammar for recognition to a recognition server at an address on the network different than the client device.

42. The method of claim 41 and further comprising:

normalizing the data indicative of the input prior to sending the data.

43. The method of claim 41 wherein the indication provides a reference to location of the grammar.

44. The method of claim 41 wherein the markup language comprises one of HTML, XHTML, cHTML, XML and WML.

45. The method of claim 41 and further comprising:

receiving a recognition result pertaining to recognized speech and associating the result with a data field on the client device.

46. The method of claim 45 wherein receiving a recognition result includes associating the recognition result to a plurality of data fields.

47. The method of claim 41 wherein the markup language comprises a scripting language.

48. The method of claim 41 wherein the markup language comprises a synchronized markup language.

50. The computer readable medium of claim 49 wherein the element is for one of speech recognition, handwriting recognition, gesture recognition, DTMF recognition and visual recognition.

51. The computer readable medium of claim 49 wherein the element comprises an indication of a grammar for recognition.

52. The computer readable medium of claim 51 wherein element pertains to receiving and processing a text string like recognition.

53. The computer readable medium of claim 52 and further comprising a second object model receiving the text string from another component.

54. The computer readable medium of claim 49 wherein the element comprises an indication for associating recognition result to a data field.

55. The computer readable medium of claim 49 wherein the element comprises an indication for associating recognition result to a plurality of data fields.

56. The computer readable medium of claim 49 wherein the markup language includes instructions for handling an event.

57. The computer readable medium of claim 56 wherein the event pertains to a time period.

58. The computer readable medium of claim 56 wherein the event pertains to whether recognition was obtained.

59. The computer readable medium of claim 56 wherein the event pertains to when a DTMF key is pressed.

60. The computer readable medium of claim 59 wherein the event pertains to call control.

61. The computer readable medium of claim 49 wherein the element pertains to audibly prompting a user.

62. The computer readable medium of claim 49 wherein the markup language comprises one of HTML, XHTML, cHTML, XML and WML.

63. The computer readable medium of claim 49 wherein the markup language comprises a scripting language.

64. The computer readable medium of claim 49 wherein the markup language comprises a synchronized markup language.

49. A computer readable medium having a markup language for execution on a client device in a client/server system, the markup language comprising an instruction indicating an object model element having an attribute or a method for recognition.

65. A computer readable medium having a markup language for execution on a client device in a client/server system, the markup language comprising an instruction indicating an object model element having an attribute or a method for audibly prompting.

66. The computer readable medium of claim 65 wherein the markup language comprises one of HTML, XHTML, cHTML, XML and WML.

67. The computer readable medium of claim 65 wherein the markup language comprises a scripting language.

68. The computer readable medium of claim 65 wherein the markup language comprises a synchronized markup language.

69. The computer readable medium of claim 65 wherein the element comprises playing an audio file.

70. The computer readable medium of claim 65 wherein the element comprises converting text-to-speech.

71. The computer readable medium of claim 70 wherein the element obtains dynamic content for converting text-to-speech.

72. The computer readable medium of claim 65 wherein one of the attributes comprises a period of time when prompting can be interrupted.

73. The computer readable medium of claim 65 wherein the markup language includes instructions for handling an event related to prompting.

74. The computer readable medium of claim 73 wherein the event pertains to whether prompting is being interrupted.

75. The computer readable medium of claim 65 wherein one of the attributes relates to rendering textual string indicative of the prompt played.