Method and system for unified speech and graphic user interfaces

• US 20050071171A1
• Filed: 09/30/2003
• Published: 03/31/2005
• Est. Priority Date: 09/30/2003
• Status: Active Grant

• Current Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Original Assignee: Motorola, Inc. (Motorola Solutions, Inc.)
• Inventors: Dvorak, Joseph L.
• Granted Patent: US 7,389,235 B2
• Time in Patent Office: Days
• US Class Current: 704/275
• CPC Class Codes: G10L 15/28 Constructional details of speech recognition systems

Abstract

A method for unifying speech user interface and graphic user interface commands includes the steps of receiving (52) user entered text via a GUI, processing (54) the user-entered text, monitoring (60) the user-entered text, adding input context (62) to the user-entered text, and, updating (74, 76, and 78) a speech recognizer with the user-entered text and semantic information. Updating the speech recognizer can include the step of accepting new text information and input context to update a speech grammar (74) and recognition vocabulary of the speech recognizer. The method can include the step of updating the GUI (72) by updating GUI directives (68) and elements (70) to maintain the GUI unified with the speech grammar. The method can further include the step of forming a window (402) enabling the display of a speech interface command as a user constructs the speech interface command using the GUI (400).

Claims

1. A method for unifying speech user interface and graphic user interface commands, comprising the steps of:

receiving grammar specifying a syntax of at least one speech command and having semantic information; and

processing the grammar to extract the semantic information for use with both a graphical user interface and a speech user interface.

2. The method of claim 1, wherein the method further comprises the step of generating graphical user interface elements corresponding to the semantic information in the grammar.

3. The method of claim 1, wherein the method further comprises the step of generating speech grammar from the grammar and semantic information.

4. The method of claim 1, wherein the method further uses the grammar and the semantic information to generate visual graphical user interface elements required to implement a set of commands contained in the grammar.

5. The method of claim 1, wherein the step of processing further comprises processing the semantic information to generate semantic directives used for parsing the grammar between the graphical user interface and the speech user interface.

6. A method for unifying speech user interface and graphic user interface commands, comprising the steps of:

receiving user entered text via a graphical user interface;

processing the user entered text via the graphical user interface;

monitoring the user entered text and adding input context to the user entered text; and

updating a speech recognizer with the user entered text and semantic information.

7. The method of claim 6, wherein the step of updating the speech recognizer comprises the step of accepting new text information and input context to augment and update a speech grammar and recognition vocabulary of the speech recognizer.

8. The method of claim 6, wherein the method further comprises the step of updating the graphical user interface.

9. The method of claim 8, wherein the step of updating the graphical user interface comprises the steps of updating graphical user interface directives and elements to maintain the graphical user interface unified with the speech grammar.

10. The method of claim 6, wherein the method further comprises forming a window for displaying a speech interface command as it is being built using the graphical user interface.

11. A system having a unified speech user interface and graphical user interface, comprising:

a display for providing an output for graphic user interface elements; and

a processor coupled to the display, the processor programmed to;

receive grammar specifying a syntax of at least one speech command and having semantic information; and

process the grammar to extract the semantic information for use with both a graphical user interface and a speech user interface.

12. The system of claim 11, wherein the processor is further programmed to generate graphical user interface elements corresponding to the semantic information in the grammar.

13. The system of claim 11, wherein the processor is further programmed to generate speech grammar from the grammar and semantic information.

14. The system of claim 11, wherein the processor is further programmed to use the grammar and the semantic information to generate visual graphical user interface elements required to implement a set of commands contained in the grammar.

15. The system of claim 11, wherein the processor is further programmed to process the semantic information to generate semantic directives used for parsing the grammar between the graphical user interface and the speech user interface.

16. A system having a unified speech user interface and graphical user interface, comprising:

a display for providing an output for graphic user interface elements; and

a processor coupled to the display, the processor programmed to;

receive user entered text via a graphical user interface;

process the user entered text via the graphical user interface;

monitor the user entered text and adding input context to the user entered text; and

update a speech recognizer with the user entered text and semantic information.

17. The system of claim 16, wherein the processor is further programmed to update the speech recognizer by accepting new text information and input context to augment and update a speech grammar and recognition vocabulary of the speech recognizer.

18. The system of claim 16, wherein the processor is further programmed to update the graphical user interface by updating graphical user interface directives and graphical user elements to maintain the graphical user interface unified with the speech grammar of the speech user interface.

19. The system of claim 16, wherein the processor is further programmed to form a window in the graphical user interface enabling the display of a speech interface command as a user constructs the speech interface command.

20. A machine readable storage, having stored thereon a computer program having a plurality of code sections executable by a machine for causing the machine to perform the steps of:

receiving grammar specifying a syntax of at least one speech command and having semantic information; and

processing the grammar to extract the semantic information for use with both a graphical user interface and a speech user interface.

21. A machine readable storage, having stored thereon a computer program having a plurality of code sections executable by a machine for causing the machine to perform the steps of:

receiving user entered text via a graphical user interface;

processing the user entered text via the graphical user interface;

monitoring the user entered text and adding input context to the user entered text; and

updating a speech recognizer with the user entered text and semantic information.