ENHANCED APPLICATION OF SPOKEN INPUT

• US 20080162138A1
• Filed: 03/17/2008
• Published: 07/03/2008
• Est. Priority Date: 03/08/2005
• Status: Active Grant

• Current Assignee: SAP SE
• Original Assignee: SAP AG (SAP SE)
• Inventors: Gurram, Rama, James, Frances
• Granted Patent: US 7,672,851 B2
• Time in Patent Office: Days
• US Class Current: 704/257
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 2015/228 of application context

Abstract

Enhanced application of spoken input, in which a single, natural language voice command is accessed. Using a repository that associates multiple operations with natural language voice commands, multiple selected operations that correspond to the received single, natural language voice command are determined and applied to a user interface.

Claims

1. A computer-implemented method comprising:

accessing a single, natural language voice command;

determining, using a repository that associates multiple operations with natural language voice commands, multiple selected operations that correspond to the received single, natural language voice command; and

applying the multiple selected operations to a user interface.

2. The method of claim 1, wherein the repository comprises an eXtensible Markup Language (XML) configuration repository.

3. The method of claim 1, wherein:

the single, natural voice command comprises a command to order a quantity of a product; and

the multiple selected operations further comprise;

a first operation operable to identify a text field within the user interface corresponding to product,

a second operation operable to select the identified text field, and

a third operation operable to enter the quantity into the selected text field.

4. The method of claim 1, wherein:

the single, natural voice command comprises a command to switch between first and second user interfaces; and

the multiple selected operations further comprise;

a first operation operable to exit the first user interface,

a second operation operable to start, after exiting the first user interface, the second user interface, and

a third operation operable to log into the started second user interface.

5. The method of claim 1, further comprising:

performing an analysis of the user interface and metadata; and

automatically generating the repository based on the analysis.

6. The method of claim 1, wherein the multiple selected operations are determined using a finite state machine.

7. The method of claim 1, wherein the single, natural language voice command further comprises a context-free voice command filtered from natural speech of a user.

8. The method of claim 1, further comprising modifying, by adding identifiers to interface elements, a non-voice enabled user interface,

wherein the multiple selected operations are applied to an interaction element specified by an added identifier in the single, natural language voice command.

9. The method of claim 1, further comprising registering the repository with an input handler.

10. The method of claim 1, further comprising:

determining, based on determining the multiple selected operations, that additional information is needed to execute the determined multiple selected operations; and

prompting a user for the additional input;

wherein the multiple selected operations are applied to the user interface based on receiving the additional input.

11. The method of claim 1, further comprising outputting feedback indicia based on applying the multiple selected operations.

12. The method of claim 1, further comprising accessing role information stored in the repository,

wherein the multiple selected operations are determined based on the accessed role information and a role of a user issuing the single, natural language voice command.

13. The method of claim 12, further comprising:

determining the role of the user; and

parsing the repository using the determined role.

14. The method of claim 1, further comprising:

parsing a spoken input;

determining whether at least a portion of the parsed spoken input represents the single, natural language voice command; and

outputting, as the single, natural language voice command, the determined portion of the parsed spoken input.

15. The method of claim 1, further comprising:

accessing a second single, natural language voice command;

determining, from the repository, additional multiple selected operations that correspond to the received second single, natural language voice command; and

applying the additional multiple selected operations to the user interface.

16. The method of claim 1, further comprising enabling disambiguation of the single, natural language voice command.

17. The method of claim 1, wherein the multiple selected operations comprise a semantic operation.

18. The method of claim 1, wherein applying the multiple selected operations to a user interface further comprises interacting with an element of the user interface using the multiple selected operations.

19. A computer program product, tangibly embodied in a machine-readable medium, the computer program product comprising instructions that, when read by a machine, operate to cause data processing apparatus to:

access a single, natural language voice command;

determine, using a repository that associates multiple operations with natural language voice commands, multiple selected operations that correspond to the received single, natural language voice command; and

apply the multiple selected operations to a user interface.

20. A device comprising:

a user interface;

a microphone configured to receive a single, natural language voice command; and

a processor configured to;

access the single, natural language voice command;

determine, using a repository that associates multiple operations with natural language voice commands, multiple selected operations that correspond to the received single, natural language voice command; and

apply the multiple selected operations to the user interface.