Method and system for unified speech and graphic user interfaces
First Claim
1. A method for unifying speech user interface and graphic user interface commands, comprising the steps of:
- receiving grammar specifying a syntax of at least one speech command and having semantic information;
processing the grammar to extract the semantic information for use with both a graphical user interface and a speech user interface;
receiving a user input to select at least one component in the graphical user interface during a navigation of the graphical user interface;
audibly receiving the at least one speech command associated with the at least one component from the speech user interface;
generating semantic directives from user supplied input text, the user input, and the at least one speech command to update a recognition vocabulary for recognizing one or more non-unified components of the graphical user interface;
parsing the grammar between the graphical user interface and the speech user interface in accordance with the semantic directives to unify the graphical user interface with the grammar from the recognition vocabulary;
visually presenting the at least one speech command in a visual prompt of the graphical user interface responsive to the step of audibly receiving; and
building a corresponding speech command for a plurality of speech commands associated with a plurality of components,wherein the corresponding speech command is a single instance of a plurality of speech commands associated with a selection of one or more components of the graphical user interface during the navigation of the graphical user interface.
4 Assignments
0 Petitions
Accused Products
Abstract
A method for unifying speech user interface and graphic user interface commands includes the steps of receiving (52) user entered text via a GUI, processing (54) the user-entered text, monitoring (60) the user-entered text, adding input context (62) to the user-entered text, and, updating (74, 76, and 78) a speech recognizer with the user-entered text and semantic information. Updating the speech recognizer can include the step of accepting new text information and input context to update a speech grammar (74) and recognition vocabulary of the speech recognizer. The method can include the step of updating the GUI (72) by updating GUI directives (68) and elements (70) to maintain the GUI unified with the speech grammar. The method can further include the step of forming a window (402) enabling the display of a speech interface command as a user constructs the speech interface command using the GUI (400).
-
Citations
20 Claims
-
1. A method for unifying speech user interface and graphic user interface commands, comprising the steps of:
-
receiving grammar specifying a syntax of at least one speech command and having semantic information; processing the grammar to extract the semantic information for use with both a graphical user interface and a speech user interface; receiving a user input to select at least one component in the graphical user interface during a navigation of the graphical user interface; audibly receiving the at least one speech command associated with the at least one component from the speech user interface; generating semantic directives from user supplied input text, the user input, and the at least one speech command to update a recognition vocabulary for recognizing one or more non-unified components of the graphical user interface; parsing the grammar between the graphical user interface and the speech user interface in accordance with the semantic directives to unify the graphical user interface with the grammar from the recognition vocabulary; visually presenting the at least one speech command in a visual prompt of the graphical user interface responsive to the step of audibly receiving; and building a corresponding speech command for a plurality of speech commands associated with a plurality of components, wherein the corresponding speech command is a single instance of a plurality of speech commands associated with a selection of one or more components of the graphical user interface during the navigation of the graphical user interface. - View Dependent Claims (2, 3, 4)
-
-
5. A method for unifying speech user interface and graphic user interface commands, comprising the steps of:
-
receiving user entered text via a graphical user interface; processing the user entered text for semantic directives via the graphical user interface; monitoring the user entered text and adding input context associated with a navigation in the graphical user interface corresponding to the user entered text; updating a speech recognizer with the user entered text for recognizing one or more non-unified components of the graphical user interface from the semantic directives; audibly receiving at least one speech command associated with the selection of the at least one component from the navigation of the speech user interface; visually presenting the at least one speech command in a visual prompt of the graphical user interface responsive to the step of audibly receiving; and building a corresponding speech command for a plurality of speech commands associated with a plurality of components and non-unified components, wherein the corresponding speech command is a single instance of a plurality of speech commands associated with a selection of one or more components of the graphical user interface during the navigation of the graphical user interface. - View Dependent Claims (6, 7, 8, 9)
-
-
10. A system having a unified speech user interface and graphical user interface, comprising:
-
a display for providing an output for graphic user interface elements; and a processor coupled to the display, the processor programmed to; receive grammar from a user input text specifying a syntax of at least one speech command and having semantic information; process the grammar from the text to extract the semantic information for use with both a graphical user interface and a speech user interface; receive a user input to select at least one component in the graphical user interface during a navigation of the graphical user interface; audibly receive the at least one speech command associated with the at least one component from the speech user interface; visually present the at least one speech command in a visual prompt of the graphical user interface responsive to the step of audibly receiving; and build a corresponding speech command for a plurality of speech commands associated with a plurality of non-unified components in accordance with the semantic information, updating a recognition vocabulary using the semantic information for recognizing the plurality of non-unified components of the graphical user interface; wherein the corresponding speech command is a single instance of a plurality of speech commands associated with a selection of one or more components of the graphical user interface during the navigation of the graphical user interface. - View Dependent Claims (11, 12, 13, 14)
-
-
15. A system having a unified speech user interface and graphical user interface, comprising:
-
a display for providing an output for graphic user interface elements; and a processor coupled to the display, the processor programmed to; receive user entered text via a graphical user interface; process the user entered text via the graphical user interface; monitor the user entered text and adding input context associated with a selection of at least one unified component in the graphical user interface to the user entered text during a navigation of the graphical user interface; and generate semantic directives from the user entered text to update a recognition vocabulary for recognizing one or more non-unified components of the graphical user interface update a speech recognizer with the user entered text and semantic information for recognizing a plurality of unified and non-unified components of the graphical user interface; audibly receive at least one speech command associated with the selection of at least one component from the speech user interface; visually present the at least one speech command in a visual prompt of the graphical user interface responsive to the step of audibly receiving; and build a corresponding speech command for a plurality of speech commands associated with a plurality of unified components and non-unified components, wherein the corresponding speech command is a single instance of a plurality of speech commands associated with a selection of one or more components of the graphical user interface during the navigation of the graphical user interface. - View Dependent Claims (16, 17, 18)
-
-
19. A machine readable storage, having stored thereon a computer program having a plurality of code sections executable by a machine for causing the machine to perform the steps of:
-
receiving grammar specifying a syntax of at least one speech command and having semantic information; and processing the grammar to extract the semantic information for use with both a graphical user interface and a speech user interface receiving a user input to select at least one component in the graphical user interface during a navigation of the graphical user interface; audibly receiving the at least one speech command associated with the at least one component from the speech user interface; visually presenting the at least one speech command in a visual prompt of the graphical user interface responsive to the step of audibly receiving; updating a speech recognizer with the user entered text and semantic information for recognizing a plurality of non-unified components of the graphical user interface; and building a corresponding speech command for a plurality of speech commands associated with a plurality of components and non-unified components, wherein the corresponding speech command is a single instance of a plurality of speech commands associated with a selection of one or more components of the graphical user interface during the navigation of the graphical user interface.
-
-
20. A machine readable storage, having stored thereon a computer program having a plurality of code sections executable by a machine for causing the machine to perform the steps of:
-
receiving user entered text via a graphical user interface; processing the user entered text via the graphical user interface; monitoring the user entered text and adding input context associated with a selection of at least one component in the graphical user interface to the user entered text; updating a speech recognizer with the user entered text and semantic information for recognizing a plurality of unified and non-unified components of the graphical user interface; audibly receiving at least one speech command associated with the selection of the at least one component from the speech user interface; visually presenting the at least one speech command in a visual prompt of the graphical user interface responsive to the step of audibly receiving; and building a corresponding speech command for a plurality of speech commands associated with a plurality of unified components and non-unified components, wherein the corresponding speech command is a single instance of a plurality of speech commands associated with a selection of one or more components of the graphical user interface during the navigation of the graphical user interface.
-
Specification