XML based architecture for controlling user interfaces with contextual voice commands

• US 7,409,344 B2
• Filed: 03/08/2005
• Issued: 08/05/2008
• Est. Priority Date: 03/08/2005
• Status: Active Grant

• Current Assignee: SAP SE
• Original Assignee: SAP AG (SAP SE)
• Inventors: Gurram, Rama, James, Frances
• Primary Examiner(s): ABEBE, DANIEL DEMELASH
• Application Number:
  US11/074,082
• Publication Number:
  US 20060206336A1
• Time in Patent Office: 1,246 Days
• Field of Search: 704/251, 704/270, 704/275, 704/254, 379/88.01, 379/88.03
• US Class Current: 704/251
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 2015/228 of application context

Abstract

A voice-enabled user interface includes a first user interface. A voice extension module is associated with the first user interface and is configured to voice-enable the first user interface. The voice extension module includes a speech recognition engine, an XML configuration repository, a preprocessor, and an input handler. The XML configuration repository includes one or more XML files specifying one or more voice commands for signaling for execution of one or more semantic operations that may be performed using the first user interface. The preprocessor is configured to register with the speech recognition engine the one or more voice commands. The input handler is configured to receive a first voice command and to communicate with the preprocessor to execute a semantic operation from the one or more semantic operations that may be performed using the first user interface. The first voice command is one of the one or more voice commands registered with the speech recognition engine by the preprocessor, and the first voice command signals for execution of the semantic operation.

Claims

1. A voice-enabled user interface comprising:

a first user interface; and

a voice extension module associated with the first user interface and configured to voice-enable the first user interface, the voice extension module including;

a speech recognition engine;

an XML configuration repository that includes one or more XML files specifying one or more voice commands for signaling for execution of one or more semantic operations that may be performed using the first user interface;

a preprocessor that is configured to register with the speech recognition engine the one or more voice commands; and

an input handler that is configured to receive a first voice command and to communicate with the preprocessor to execute a semantic operation from the one or more semantic operations that may be performed using the first user interface, wherein the first voice command is one of the one or more voice commands registered with the speech recognition engine by the preprocessor, and wherein the first voice command signals for execution of the semantic operation,

wherein;

the one or more XML files included in the XML configuration repository specify one or more additional voice commands for switching to a second user interface;

the preprocessor is configured to register with the speech recognition engine the one or more additional voice commands; and

the input handler is configured to receive a second voice command and to communicate with the preprocessor to switch to the second user interface, wherein the second voice command is one of the one or more additional voice commands registered with the speech recognition engine by the preprocessor.

2. The voice-enabled user interface of claim 1 wherein:

the XML configuration repository categorizes the one or more voice commands into one or more roles of users of the first user interface; and

the preprocessor is configured to register with the speech recognition engine one or more voice commands from within the XML configuration repository that are representative of a particular one of the one or more roles, the particular role corresponding to a user of the first user interface.

3. The voice-enabled user interface of claim 1 wherein the voice extension module includes an error handler that is configured (i) to handle errors in the execution of the semantic operation that is executed in response to the first voice command, and (ii) to prompt for additional information that further specifies the semantic operation that is executed in response to the first voice command.

4. The voice-enabled user interface of claim 1 wherein the preprocessor comprises:

a parser that is configured to identify the one or more voice commands from the one or more XML files included in the XML configuration repository; and

a translator that is configured to register the one or more voice commands with the speech recognition engine such that the one or more semantic operations may be executed in response to the one or more voice commands.

5. The voice-enabled user interface of claim 1 wherein the voice extension module includes a web service interface that is configured (i) to receive the one or more XML files included in the XML configuration repository, and (ii) to store the received XML files in the XML configuration repository.

6. A voice extension module for voice-enabling a user interface comprising:

a speech recognition engine;

an XML configuration repository that includes one or more XML files specifying one or more voice commands for signaling for execution of one or more semantic operations that may be performed using a first user interface;

a preprocessor that is configured to register with the speech recognition engine the one or more voice commands; and

an input handler that is configured to receive a first voice command and to communicate with the preprocessor to execute a semantic operation from the one or more semantic operations that may be performed using the first user interface, wherein the first voice command is one of the one or more voice commands registered with the speech recognition engine by the preprocessor, and wherein the first voice command signals for execution of the semantic operation,

wherein;

the one or more XML files included in the XML configuration repository specify one or more additional voice commands for switching to a second user interface;

the preprocessor is configured to register with the speech recognition engine the one or more additional voice commands; and

the input handler is configured to receive a second voice command and to communicate with the preprocessor to switch to the second user interface, wherein the second voice command is one of the one or more additional voice commands registered with the speech recognition engine by the preprocessor.

7. The voice extension module of claim 6 wherein:

the XML configuration repository categorizes the one or more voice commands into one or more roles of users of the first user interface; and

the preprocessor is configured to register with the speech recognition engine one or more voice commands from within the XML configuration repository that are representative of a particular one of the one or more roles, the particular role corresponding to a user of the first user interface.

8. The voice extension module of claim 6 further comprising an error handler that is configured (i) to handle errors in the execution of the semantic operation that is executed in response to the first voice command, and (ii) to prompt for additional information that further specifies the semantic operation that is executed in response to the first voice command.

9. The voice extension module of claim 6 wherein the preprocessor comprises;

a parser that is configured to identify the one or more voice commands from the one or more XML files included in the XML configuration repository; and

a translator that is configured to register the one or more voice commands with the speech recognition engine such that the one or more semantic operations may be executed in response to the one or more voice commands.

10. The voice extension module of claim 6 further comprising a web service interface that is configured (i) to receive the one or more XML files included in the XML configuration repository, and (ii) to store the received XML files in the XML configuration repository.

11. A method for enabling a user interface to be controlled with voice commands, the method comprising:

accessing an XML configuration repository that specifies one or more voice commands for execution of one or more semantic operations that may be performed using a first user interface for a first application, each voice command corresponding to at least one of the semantic operations;

identifying at least one of the voice commands from the XML configuration repository;

registering the identified voice command with a speech recognition engine and an input handler to enable voice control of the first user interface;

performing a particular one of the one or more semantic operations in response to a first voice command, wherein the first voice command is the voice command registered with the speech recognition engine and the input handler, and wherein the first voice command corresponds to the particular semantic operation;

identifying at least one additional voice command from the XML configuration repository, the at least one additional voice command corresponding to one or more switches to a second user interface for a second application;

registering the at least one additional voice command with the speech recognition engine and the input handler to enable switching to the second user interface; and

performing a particular one of the switches to the second user interface in response to a second voice command, wherein the second voice command is the additional voice command registered with the speech recognition engine and the input handler, and wherein the second voice command corresponds to the particular switch to the second user interface.

12. The method of claim 11 further comprising identifying a role of a user of the first user interface, wherein the one or more voice commands are organized within the XML configuration repository into one or more roles of users of the first user interface, and wherein identifying at least one of the voice commands comprises identifying one or more voice commands that correspond to the identified role from the XML configuration repository.

13. The method of claim 11 wherein identifying the one or more voice commands from the XML configuration repository comprises parsing one or more XML files included in the XML configuration repository to identify the one or more voice commands.

14. The method of claim 11 further comprising handling errors in the execution of the particular semantic operation that is performed in response to the first voice command with an error handler.

15. A voice-enabled user interface comprising:

first and second user interfaces; and

a voice extension module associated with the first user interface and configured to voice-enable the first user interface, the voice extension module including;

a speech recognition engine;

an XML configuration repository that includes one or more XML files specifying one or more voice commands for signaling for execution of one or more semantic operations that may be performed using the first user interface;

a preprocessor that is configured to register with the speech recognition engine the one or more voice commands; and

an input handler that is configured to receive a first voice command and to communicate with the preprocessor to execute a semantic operation from the one or more semantic operations that may be performed using the first user interface, wherein the first voice command is one of the one or more voice commands registered with the speech recognition engine by the preprocessor, and wherein the first voice command signals for execution of the semantic operation,

wherein;

the one or more XML files included in the XML configuration repository specify one or more additional voice commands for signaling for execution of one or more semantic operations that may be performed using the second user interface;

the preprocessor is configured to register with the speech recognition engine the one or more additional voice commands; and

the input handler is configured to receive a second voice command and to communicate with the preprocessor to execute a semantic operation from the one or more semantic operations that may be performed using the second user interface, wherein the second voice command is one of the one or more additional voice commands registered with the speech recognition engine by the preprocessor, and wherein the second voice command signals for execution of the semantic operation.

16. A voice extension module for voice-enabling a user interface comprising:

a speech recognition engine;

an XML configuration repository that includes one or more XML files specifying one or more voice commands for signaling for execution of one or more semantic operations that may be performed using a first user interface;

a preprocessor that is configured to register with the speech recognition engine the one or more voice commands; and

an input handler that is configured to receive a first voice command and to communicate with the preprocessor to execute a semantic operation from the one or more semantic operations that may be performed using the first user interface, wherein the first voice command is one of the one or more voice commands registered with the speech recognition engine by the preprocessor, and wherein the first voice command signals for execution of the semantic operation,

wherein;

the one or more XML files included in the XML configuration repository specify one or more additional voice commands for signaling for execution of one or more semantic operations that may be performed using a second user interface;

the preprocessor is configured to register with the speech recognition engine the one or more additional voice commands; and

the input handler is configured to receive a second voice command and to communicate with the preprocessor to execute a semantic operation from the one or more semantic operations that may be performed using the second user interface, wherein the second voice command is one of the one or more additional voice commands registered with the speech recognition engine by the preprocessor, and wherein the second voice command signals for execution of the semantic operation.

17. A method for enabling a user interface to be controlled with voice commands, the method comprising:

accessing an XML configuration repository that specifies one or more voice commands for execution of one or more semantic operations that may be performed using a first user interface for a first application, each voice command corresponding to at least one of the semantic operations;

identifying at least one of the voice commands from the XML configuration repository;

registering the identified voice command with a speech recognition engine and an input handler to enable voice control of the first user interface;

performing a particular one of the one or more semantic operations in response to a first voice command, wherein the first voice command is the voice command registered with the speech recognition engine and the input handler, and wherein the first voice command corresponds to the particular semantic operation,

identifying at least one additional voice command from the XML configuration repository, the at least one additional voice command corresponding to one or more semantic operations that may be performed using a second user interface for a second application;

registering the at least one additional voice command with the speech recognition engine and the input handler to enable voice control of the second user interface; and

performing a particular one of the semantic operations that may be performed using the second user interface in response to a second voice command, wherein the second voice command is the additional voice command registered with the speech recognition engine and the input handler, and wherein the second voice command corresponds to the particular semantic operation.