Voice-control for a user interface

• US 6,882,974 B2
• Filed: 08/28/2002
• Issued: 04/19/2005
• Est. Priority Date: 02/15/2002
• Status: Expired due to Term

• Current Assignee: SAP SE
• Original Assignee: SAP AG (SAP SE)
• Inventors: James, Frankie, Gurram, Rama, Swan, Richard, Roelands, Jeff
• Primary Examiner(s): Dorvil, Richemond
• Assistant Examiner(s): ARMSTRONG, ANGELA A
• Application Number:
  US10/229,266
• Publication Number:
  US 20030158736A1
• Time in Patent Office: 965 Days
• Field of Search: 704/270.1, 704/270, 704/257, 704/235, 704/275, 704/251, 704/252, 704/9, 704/2, 345/746
• US Class Current: 704/270.1
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 2015/228 of application context

Abstract

Method and systems to voice-enable a user interface using a voice extension module are provided. A voice extension module includes a preprocessor, a speech recognition engine, and an input handler. The voice extension module receives user interface information, such as, a hypertext markup language (HTML) document, and voice-enables the document so that a user may interact with any user interface elements using voice commands.

Claims

1. A voice-enabled user interface comprising:

user interface elements; and

a speech recognition engine that receives voice input identifying a target user interface element,

wherein the voice-enabled user interface resolves non-verbal ambiguities in associating the received voice input with the target user interface element by displaying a representational enumerated label associated positionally with the target user interface element after the voice input is received.

2. The voice-enabled user interface of claim 1 wherein the user interface elements include one or more from the list consisting of:

combo box;

text field;

list box;

button;

text area;

check box; and

radio buttons.

3. The voice-enabled user interface of claim 1 wherein the speech recognition engine receives voice input identifying a category of user interface elements.

4. The voice-enabled user interface of claim 1 wherein the representational enumerated labels are semi-transparent.

5. The voice-enabled user interface of claim 1 wherein the representational enumerated labels are opaque.

6. The voice-enabled user interface of claim 1 wherein the voice-enabled user interface resolves the non-verbal ambiguities using implicit scoping.

7. The voice-enabled user interface of claim 6 wherein the user interface elements are divided into one or more groups, and implicit scoping includes prioritizing the one or more groups.

8. The voice-enabled user interface of claim 7 wherein each of the one or more groups represents a screen area.

9. The voice-enabled user interface of claim 8 wherein the visual cues include a colored highlight.

10. The voice-enabled user interface of claim 9 wherein the colored highlight surrounds a user interface element.

11. The voice-enabled user interface of claim 1 wherein the voice-enabled user interface resolves the non-verbal ambiguities using visual cues.

12. A representational enumerated label for resolving non-verbal ambiguities in a voice-enabled interface, the label comprising:

a unique identifier;

an association to a corresponding user interface element; and

a graphical representation presented in the voice-enabled interface to show the association to the corresponding user interface element;

wherein the unique identifier may be displayed after a voice input is recognized to resolve non-verbal ambiguities in associating the recognized voice input with the user interface element.

13. The representational enumerated label of claim 12 wherein the unique identifier is a number.

14. The representational enumerated label of claim 12 wherein the unique identifier is a letter.

15. The representational enumerated label of claim 12 wherein the unique identifier is an icon.

16. The representational enumerated label of claim 12 wherein the unique identifier is a character.

17. The representational enumerated label of claim 12 wherein the unique corresponding user interface element includes one or more from the group consisting of:

combo box;

text field;

list box;

button;

text area;

check box; and

radio buttons.

18. The representational enumerated label of claim 12 wherein the graphical representation is semi-transparent.

19. The representational enumerated label of claim 12 wherein the graphical representation is opaque.

20. The representational enumerated label of claim 12 wherein the graphical representation is overlaid on the corresponding user interface element.

21. A method for resolving target ambiguity in a voice-enabled user interface comprising:

receiving first voice input that identifies more than one potential target user interface element based on an ambiguity of the user interface to present a target ambiguity;

displaying representational enumerated labels corresponding to each potential target user interface element after the first voice input is received, each representational enumerated label including positional association to a corresponding target user interface element and a unique identifier; and

receiving second voice input including the unique identifier of one of the representational enumerated labels to resolve the target ambiguity.

22. The method of claim 21 wherein receiving the first voice input includes receiving voice input identifying a category of ambiguously indicated user interface elements.

23. The method of claim 22 wherein the category of ambiguously indicated user interface elements includes one or more from the group consisting of:

combo box;

text field;

list box;

button;

text area;

check box; and

radio buttons.

24. The method of claim 21 wherein displaying representational enumerated labels corresponding to each potential target user interface element includes overlaying a graphical representation on each potential target user interface element.

25. The method of claim 24 wherein the graphical representation is semi-transparent.

26. The method of claim 24 wherein the graphical representation is opaque.

27. The method of claim 21 wherein each of the representational enumerated labels further includes;

a graphical representation presented in the voice-enabled interface to show the spatial association to the corresponding target user interface element.

28. The method of claim 27 wherein the unique identifier is a number.

29. The method of claim 27 wherein the unique identifier is letter.

30. The method of claim 27 wherein the graphical representation is semi-transparent.

31. The method of claim 27 wherein the graphical representation is opaque.

32. A voice-enabled user interface comprising:

a user interface element configured to influence voice input of a user by providing a visual cue indicative to the user of a grammar associated with the user interface element;

an input handler that enables the user to specify the grammar to be associated with the user interface element;

a data store configured to store the association between the user interface element and the grammar; and

a speech recognition engine that receives voice input identifying the user interface element, queries the data store to determine the grammar associated with the user interface element, and processes data input using the determined grammar.

33. The voice-enabled user interface of claim 32 wherein the data store is accessible by a single user.

34. The voice-enabled user interface of claim 32 wherein the data store is accessible by multiple users.

35. The voice-enabled user interface of claim 34 wherein the user may use the input handler to associate a data entry field of the user interface element with the grammar in the data store for access by the multiple users.

36. The voice-enabled user interface of claim 32 wherein the voice-enabled user interface displays visual cues with the target user interface element.

37. The voice-enabled user interface of claim 32 wherein the visual cue identifies a data type associated with the user interface element.

38. The voice-enabled user interface of claim 36 wherein the visual cues identify a grammar associated with the target user interface element.

39. A voice-enabled user interface comprising:

user interface elements; and

a speech recognition engine that receives voice input identifying a target user interface element,

wherein the voice-enabled user interface resolves ambiguities in associating the received voice input with the target user interface element using implicit scoping.

40. The voice-enabled user interface of claim 39 wherein the user interface elements are divided into one or more groups, and implicit scoping includes prioritizing the one or more groups.

41. The voice-enabled user interface of claim 40 wherein each of the one or more groups represents a screen area.