Controlling user interfaces with contextual voice commands

• US 20060136221A1
• Filed: 12/22/2004
• Published: 06/22/2006
• Est. Priority Date: 12/22/2004
• Status: Active Grant

• Current Assignee: SAP SE
• Original Assignee: SAP SE
• Inventors: Gurram, Rama, James, Frances
• Granted Patent: US 8,788,271 B2
• Time in Patent Office: Days
• US Class Current: 704/275
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

One or more voice-enabled user interfaces include a user interface, and a voice extension module associated with the user interface. The voice extension module is configured to voice-enable the user interface and includes a speech recognition engine, a preprocessor, and an input handler. The preprocessor registers with the speech recognition engine one or more voice commands for signaling for execution of one or more semantic operations that may be performed using a first user interface. The input handler receives a first voice command and communicates with the preprocessor to execute a semantic operation that is indicated by the first voice command. The first voice command is one of the voice commands registered with the speech recognition engine by the preprocessor.

Claims

1. A voice-enabled user interface comprising:

a first user interface; and

a voice extension module associated with the first user interface and configured to voice-enable the user interface, the voice extension module including;

a speech recognition engine;

a preprocessor that registers with the speech recognition engine one or more voice commands for signaling for execution of one or more semantic operations that may be performed using the first user interface; and

an input handler that receives a first voice command and communicates with the preprocessor to execute a semantic operation that is indicated by the first voice command, the first voice command being one of the voice commands registered with the speech recognition engine by the preprocessor.

2. The voice-enabled user interface of claim 1 wherein:

the preprocessor registers with the speech recognition engine one or more additional voice commands for switching to a second user interface; and

the input handler receives a second voice command and communicates with the preprocessor to switch to the second user interface, the second voice command being one of the additional voice commands registered with the speech recognition engine by the preprocessor.

3. The voice-enabled user interface of claim 1 further comprising a second user interface, wherein:

the preprocessor registers with the speech recognition engine one or more additional voice commands for signaling for execution of one or more semantic operations that may be performed using the second user interface; and

the input handler receives a second voice command and communicates with the preprocessor to execute a semantic operation that is indicated by the second voice command, the second voice command being one of the additional voice commands registered with the speech recognition engine by the preprocessor.

4. The voice-enabled user interface of claim 1 wherein:

the preprocessor registers with the speech recognition engine one or more additional voice commands for controlling one or more graphical elements of the first user interface; and

the input handler receives a second voice command and communicates with the preprocessor to control a graphical element of the first user interface that is indicated by the second voice command, the second voice command being one of the additional voice commands registered with the speech recognition engine by the preprocessor.

5. The voice-enabled user interface of claim 1 wherein the input handler prompts for additional information that further specifies the semantic operation that is executed in response to the first voice command.

6. The voice-enabled user interface of claim 1 wherein the preprocessor comprises:

a generator configured to identify one or more semantic operations that may be performed with the user interface; and

a translator configured to register with the speech recognition engine one or more voice commands for signaling for execution of the one or more identified semantic operations.

7. The voice-enabled user interface of claim 1 wherein the user interface is at least one from a group including a hypertext markup language (HTML) document presented in a web browser, a standalone application, and a user interface for a web services application.

8. A voice extension module for voice-enabling a user interface comprising:

a speech recognition engine;

a preprocessor that registers with the speech recognition engine one or more voice commands for signaling for execution of one or more semantic operations that may be performed using a user interface; and

an input handler that receives a first voice command and communicates with the preprocessor to execute a semantic operation that is indicated by the first voice command using the user interface, the first voice command being one of the voice commands registered with the speech recognition engine by the preprocessor.

9. The voice extension module of claim 8 wherein:

the preprocessor registers with the speech recognition engine one or more additional voice commands for switching to a second user interface; and

the input handler receives a second voice command and communicates with the preprocessor to switch to the second user interface, the second voice command being one of the additional voice commands registered with the speech recognition engine by the preprocessor.

10. The voice extension module of claim 8 wherein:

the preprocessor registers with the speech recognition engine one or more additional voice commands for signaling for execution of one or more semantic operations that may be performed using a second user interface; and

the input handler receives a second voice command and communicates with the preprocessor to execute a semantic operation that is indicated by the second voice command, the second voice command being one of the additional voice commands registered with the speech recognition engine by the preprocessor.

11. The voice extension module of claim 8 wherein:

the preprocessor registers with the speech recognition engine one or more additional voice commands for controlling one or more graphical elements of the first user interface; and

the input handler receives a second voice command and communicates with the preprocessor to control a graphical element of the first user interface that is indicated by the second voice command, the second voice command being one of the additional voice commands registered with the speech recognition engine by the preprocessor.

12. The voice extension module of claim 8 wherein the input handler prompts for additional information that further specifies the semantic operation that is performed in response to the first voice command.

13. The voice extension module of claim 8 wherein the preprocessor comprises;

a generator configured to identify one or more semantic operations that may be performed with the user interface; and

a translator configured to register with the speech recognition engine one or more voice commands for signaling for execution of the one or more identified semantic operations.

14. A method for enabling a user interface to be controlled with voice commands, the method comprising:

accessing information describing a first user interface that enables interaction with a first application;

identifying one or more semantic operations that may be performed with the first user interface;

registering one or more voice commands with a speech recognition engine to enable voice control of the first user interface, each voice command corresponding to one of the semantic operations; and

performing one of the semantic operations in response to a first voice command, the first voice command being one of the voice commands registered with the speech recognition engine, the performed semantic operation corresponding to the first voice command.

15. The method of claim 14 further comprising:

registering one or more additional voice commands with a speech recognition engine to enable switching to a second user interface of a second application, each additional voice command corresponding to a switch to the second user interface; and

performing, in response to a second voice command that is one of the additional voice commands registered with the speech recognition engine and that corresponds to a switch to the second application, the switch to the second user interface corresponding to the second voice command.

16. The method of claim 14 further comprising:

registering one or more additional voice commands with a speech recognition engine to enable voice control of a second user interface, each additional voice command corresponding to a semantic operation that may be performed with the second user interface; and

performing one of the semantic operations in response to a second voice command, the second voice command being one of the additional voice commands registered with the speech recognition engine, the performed semantic operation corresponding to the second voice command.

17. The method of claim 14 further comprising:

identifying one or more graphical elements of the first user interface that may be controlled;

registering one or more additional voice commands with a speech recognition engine to enable control of the identified graphical elements of the first user interface, each additional voice command corresponding to at least one of the identified graphical elements; and

controlling one of the identified graphical elements in response to a second voice command, the second voice command being one of the additional voice commands registered with the speech recognition engine, the controlled graphical element corresponding to the second voice command.

18. The method of claim 14 further comprising prompting for additional information that further specifies the semantic operation that is performed in response to the first voice command.

19. The method of claim 18 further comprising clarifying the first voice command such that the first voice command corresponds only to the semantic operation that is performed in response to the first voice command.

20. The method of claim 14 further comprising providing feedback indicating that semantic operation indicated by the first voice command has been performed.