ENABLING GRAMMARS IN WEB PAGE FRAME

• US 20110047452A1
• Filed: 11/02/2010
• Published: 02/24/2011
• Est. Priority Date: 12/06/2006
• Status: Active Grant

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Cross, Charles W. JR., McCobb, Gerald M., Ativanichayaphong, Soonthorn
• Granted Patent: US 8,073,692 B2
• Time in Patent Office: Days
• US Class Current: 715/234
• CPC Class Codes: G06F 16/9574 of access to content, e.g. by caching
• CPC Class Codes: G06F 16/986 Document structures and storage, e.g. HTML extensions
• CPC Class Codes: G10L 15/19 Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules
• CPC Class Codes: H04M 2201/40 using speech recognition speech recognition per se G10L15/00
• CPC Class Codes: H04M 3/4938 comprising a voice browser which renders and interprets, e.g. VoiceXML

Abstract

Enabling grammars in web page frames, including receiving, in a multimodal application on a multimodal device, a frameset document, where the frameset document includes markup defining web page frames; obtaining by the multimodal application content documents for display in each of the web page frames, where the content documents include navigable markup elements; generating by the multimodal application, for each navigable markup element in each content document, a segment of markup defining a speech recognition grammar, including inserting in each such grammar markup identifying content to be displayed when words in the grammar are matched and markup identifying a frame where the content is to be displayed; and enabling by the multimodal application all the generated grammars for speech recognition.

Claims

1. A method of enabling grammars in web page frames, the method comprising:

receiving, in a multimodal application on a multimodal device, a frameset document, the frameset document comprising markup defining web page frames;

obtaining, by the multimodal application, content documents for display in each of the web page frames, the content documents comprising navigable markup elements;

generating, by the multimodal application, for each navigable markup element in each content document, a segment of markup that defines a speech recognition grammar and identifies content to be displayed when words in the grammar are matched; and

enabling, by the multimodal application, all of the generated grammars so that any of the generated grammars is usable for speech recognition at any one time.

2. The method of claim 1, wherein the web page frames are organized in a hierarchy according to one or more framesets, the hierarchy characterized by a topmost frame and one or more child frames, and wherein obtaining the content documents further comprises iteratively obtaining, for the topmost frame and for each child frame, a separate content document for display in each frame.

3. The method of claim 1, wherein enabling the generated grammars comprises:

dynamically generating a markup language fragment specifying a grammar; and

providing the markup language fragment to an automated voice markup language interpreter.

4. The method of claim 1, wherein:

the multimodal device further comprises an automated voice markup language interpreter; and

enabling the generated grammars further comprises providing the grammars to the automated voice markup language interpreter by one or more application programming interface (‘

API’

) calls from the multimodal application to the automated voice markup language interpreter.

5. The method of claim 1, wherein:

the multimodal device is coupled for data communications to a voice server, the voice server comprising an automated voice markup language interpreter; and

enabling all the generated grammars further comprises providing the grammars to the automated voice markup language interpreter via one or more data communications protocol messages from the multimodal device to the automated voice markup language interpreter on the voice server.

6. The method of claim 1, further comprising:

providing, by the multimodal application to an automated voice markup language interpreter, speech for recognition from a user;

matching, by the automated voice markup language interpreter with an enabled grammar, at least part of the speech for recognition; and

returning, from the automated voice markup language interpreter to the multimodal application, an event indicating an instruction representative of the matched speech.

7. A multimodal device, comprising a multimodal application, the multimodal device comprising at least one computer processor programmed to:

receive, in the multimodal application, a frameset document, the frameset document comprising markup defining web page frames;

obtain content documents for display in each of the web page frames, the content documents comprising navigable markup elements; and

generate, for each navigable markup element in each content document, a segment of markup that defines a speech recognition grammar, identifies content to be displayed when words in the grammar are matched, and identifies a frame where the content is to be displayed.

8. The multimodal device of claim 7, wherein the web page frames are organized in a hierarchy according to one or more framesets, the hierarchy characterized by a topmost frame and one or more child frames, and the at least one computer processor is programmed to obtain the content documents by iteratively obtaining, for the topmost frame and for each child frame, a separate content document for display in each frame.

9. At least one computer-readable storage medium having instructions encoded thereon which, when executed, perform a method comprising:

receiving, in a multimodal application on a multimodal device, a frameset document, the frameset document comprising markup defining web page frames;

obtaining, by the multimodal application, content documents for display in each of the web page frames, the content documents comprising navigable markup elements;

generating, by the multimodal application, for each navigable markup element in each content document, a segment of markup that defines a speech recognition grammar and identifies content to be displayed when words in the grammar are matched; and

enabling, by the multimodal application, all the generated grammars so that any of the generated grammars is usable for speech recognition at any one time.

10. The at least one computer-readable storage medium of claim 9, wherein:

the web page frames are organized in a hierarchy according to one or more framesets, the hierarchy characterized by a topmost frame and one or more child frames; and

obtaining at least two content documents further comprises iteratively obtaining, for the topmost frame and for each child frame, a separate content document for display in each frame.

11. The at least one computer-readable storage medium of claim 9, wherein enabling the generated grammars comprises:

dynamically generating a markup language fragment specifying a grammar; and

providing the markup language fragment to an automated voice markup language interpreter.

12. The at least one computer-readable storage medium of claim 9, wherein:

the multimodal device further comprises an automated voice markup language interpreter; and

enabling the generated grammars further comprises providing the grammars to the automated voice markup language interpreter via one or more application programming interface (‘

API’

) calls from the multimodal application to the automated voice markup language interpreter.

13. The at least one computer-readable storage medium of claim 9, wherein the multimodal device is coupled for data communications to a voice server, the voice server comprising an automated voice markup language interpreter, and wherein enabling the generated grammars comprises providing the grammars to the automated voice markup language interpreter via one or more data communications protocol messages from the multimodal device to the automated voice markup language interpreter on the voice server.

14. The at least one computer-readable storage medium of claim 9, wherein the method comprises:

providing, by the multimodal application to an automated voice markup language interpreter, speech for recognition from a user;

matching, by the automated voice markup language interpreter with an enabled grammar, at least part of the speech for recognition; and

returning, from the automated voice markup language interpreter to the multimodal application, an event indicating an instruction representative of the matched speech.