Method and apparatus for implementing a speech interface for a GUI

• US 7,036,080 B1
• Filed: 11/30/2001
• Issued: 04/25/2006
• Est. Priority Date: 11/30/2001
• Status: Expired due to Term

• Current Assignee: SAP SE
• Original Assignee: Sap Labs, Inc.
• Inventors: James, Frankie, Roelands, Jeff
• Primary Examiner(s): Hailu, Tadesse
• Application Number:
  US09/997,704
• Time in Patent Office: 1,607 Days
• Field of Search: 345727-729, 345716-717, 345/866, 345/865, 345/789, 704/275, 704/270, 704/260, 704/276, 715/520, 715/516, 715/530, 715/716, 715/727, 715/728, 715/764, 715/765, 715/766, 715/767
• US Class Current: 715/728
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

A method and apparatus for providing speech control to a graphical user interface (GUI) divide a GUI into a plurality of screen areas; assign the screen areas priorities; receive a first audio input relating to the selection of one of the objects in the interface; determine the one of the screen areas having the highest priority and including a first object matching the first audio input; and select the first object in the determined screen area if the determined screen area only contains one object matching the first audio input. The method and apparatus also select one of the objects that matches the first audio input in the determined screen area if the determined screen area contains more than one object that matches the first audio input.

Claims

1. A method for providing speech control to a graphical user interface (GUI) containing objects to be selected, the method comprising:

dividing the GUI into a plurality of screen areas;

assigning priorities to the screen areas;

receiving a first audio input relating to the selection of one of the objects in the GUI;

determining the one of the screen areas that has the highest priority and includes a first object matching the first audio input; and

selecting the first object, if the determined screen area only contains one object matching the first audio input; and

using a second input to select one of the objects that matches the first audio input in the determined screen area, if the determined screen area contains more than one object that matches the first audio input.

2. The method of claim 1, wherein using a second input includes:

marking each object matching the first audio input;

receiving a second audio input relating to the selection of one of the marked objects; and

selecting the marked object that best matches the second audio input.

3. The method of claim 2, further comprising removing the markers.

4. The method of claim 2, wherein marking the objects includes marking the objects with icons.

5. The method of claim 4, wherein marking the objects with the icons includes marking the objects with semi-transparent icons.

6. The method of claim 5, wherein marking the objects with semi-transparent icons includes marking the objects with semi-transparent icons that overlap the objects.

7. The method of claim 6, wherein marking the objects with semi-transparent icons that overlap the objects includes marking the objects with semi-transparent icons that have numbered labels.

8. The method of claim 1, wherein assigning priorities to the screen areas includes assigning priorities to the screen areas based on usage.

9. The method of claim 1, wherein assigning priorities to the screen areas includes assigning adjustable priorities to the screen areas.

10. The method of claim 1, further comprising indicating the one of the screen areas having the highest priority.

11. The method of claim 10, wherein indicating the highest priority screen area includes indicating the highest priority screen area with a visual output.

12. The method of claim 11, wherein indicating the highest priority screen area with visual output includes indicating the highest priority screen area with a visual output that includes highlighting.

13. The method of claim 1, further comprising confirming the receipt of the first audio input.

14. The method of claim 13, wherein confirming the receipt of the first audio input includes confirming the receipt of the first audio input with an audio output.

15. A method for using a graphical user interface (GUI), the method comprising:

receiving a first audio input relating to the selection of one of the objects in the GUI;

determining the one of the screen areas that has the highest priority and includes a first object matching the first audio input;

selecting the first object, if the determined screen area only contains one object matching the first audio input; and

using a second input to select one of the objects that matches the first audio input in the determined screen area, if the determined screen area contains more than one object that matches the first audio input.

16. An apparatus for providing speech control to a graphical user interface (GUI) containing objects to be selected, the apparatus comprising:

at least one memory having program instructions; and

at least one processor configured to use the program instructions to perform the operations of;

dividing the GUI into a plurality of screen areas;

assigning priorities to the screen areas;

receiving a first audio input relating to the selection of one of the objects in the GUI;

determining the one of the screen areas that has the highest priority and includes a first object matching the first audio input;

selecting the first object, if the determined screen area only contains one object matching the first audio input; and

,

using a second input to select one of the objects that matches the first audio input in the determined screen area, if the determined screen area contains more than one object that matches the first audio input.

17. The apparatus of claim 16, wherein using a second input includes:

marking each object matching the first audio input;

receiving a second audio input relating to the selection of one of the marked objects; and

selecting the marked object that best matches the second audio input.

18. The apparatus of claim 17, wherein the operations further comprise removing the markers.

19. The apparatus of claim 17, wherein marking the objects includes marking the objects with icons.

20. The apparatus of claim 19, wherein marking the objects with the icons includes marking the objects with semi-transparent icons.

21. The apparatus of claim 20, wherein marking the objects with semi-transparent icons includes marking the objects with semi-transparent icons that overlap the objects.

22. The apparatus of claim 21, wherein marking the objects with semi-transparent icons that overlap the objects includes marking the objects with semi-transparent icons that have numbered labels.

23. The apparatus of claim 16, wherein assigning priorities to the screen areas includes assigning priorities to the screen areas based on usage.

24. The apparatus of claim 16, wherein assigning priorities to the screen areas includes assigning adjustable priorities to the screen areas.

25. The apparatus of claim 16, wherein the operations further comprise indicating the one of the screen areas having the highest priority.

26. The apparatus of claim 25, wherein indicating the highest priority screen area includes indicating the highest priority screen area with a visual output.

27. The apparatus of claim 26, wherein indicating the highest priority screen area with visual output includes indicating the highest priority screen area with a visual output that includes highlighting.

28. The apparatus of claim 16, wherein the operations further comprise confirming the receipt of the first audio input.

29. The apparatus of claim 28, wherein confirming the receipt of the first audio input includes confirming the receipt of the first audio input with an audio output.

30. An apparatus for providing speech control to a graphical user interface (GUI)

containing objects to be selected, the apparatus comprising:

at least one memory having program instructions; and

at least one processor configured to use the program instructions to perform the operations of;

receiving a first audio input relating to the selection of one of the objects in the GUI;

determining the one of the screen areas that has the highest priority and includes a first object matching the first audio input;

selecting the first object, if the determined screen area only contains one object matching the first audio input; and

using a second input to select one of the objects that matches the first audio input in the determined screen area, if the determined screen area contains more than one object that matches the first audio input.

31. A system for providing speech control to a graphical user interface (GUI) containing objects to be selected, comprising:

means for dividing the GUI into a plurality of screen areas;

means for assigning priorities to the screen areas;

means for receiving a first audio input relating to the selection of one of the objects in the GUI;

means for determining the one of the screen areas that has the highest priority and includes a first object matching the first audio input;

means for selecting the first object, if the determined screen area only contains one object matching the first audio input, and

means for using a second input to select one of the objects that matches the first audio input in the determined screen area, if the determined screen area contains more than one object that matches the first audio input.

32. The system of claim 31, wherein the means for using a second input includes:

means for marking each object matching the first audio input;

means for receiving a second audio input relating to the selection of one of the marked objects; and

means for selecting the marked object that best matches the second audio input.

33. The system of claim 32, further comprising means for removing the markers.

34. The system of claim 32, wherein the markers include icons.

35. The system of claim 34, wherein the icons are semi-transparent.

36. The system of claim 35, wherein the icons overlap the objects matching the audio input.

37. The system of claim 36, wherein the icons includes numbered labels.

38. The system of claim 31, wherein the priorities of the screen areas are based on usage.

39. The system of claim 31, wherein the priorities of the screen areas are adjustable.

40. The system of claim 31, further comprising means for indicating the one of the screen areas having the highest priority.

41. The system of claim 40, wherein the means for indicating the highest priority screen area includes means for indicating the highest priority screen area with a visual output.

42. The system of claim 41, wherein the means for indicating the highest priority screen area includes means for indicating the highest priority screen area with a visual output that includes highlighting.

43. The system of claim 31, further comprising means for confirming the receipt of the first audio input.

44. The system of claim 43, wherein the means for confirming the receipt of the first audio input includes means for generating an audio output.

45. A system for providing speech control to a graphical user interface (GUI) containing objects to be selected, comprising:

means for receiving a first audio input relating to the selection of one of the objects in the GUI;

means for determining the one of the screen areas that has the highest priority and includes a first object matching the first audio input;

means for selecting the first object, if the determined screen area only contains one object matching the first audio input; and

means for using a second input to select one of the objects that matches the first audio input in the determined screen area, if the determined screen area contains more than one object that matches the first audio input.

46. A computer readable medium containing instructions for controlling a computer system to perform a method for providing speech control to a graphical user interface (GUI) containing objects to be selected, the method comprising:

dividing the GUI into a plurality of screen areas;

assigning Priorities to the screen areas;

receiving a first audio input relating to the selection of one of the objects in the GUI;

determining the one of the screen areas that has the highest priority and includes a first object matching the first audio input;

selecting the first object, if the determined screen area only contains one object matching the first audio input; and

using a second input to select one of the objects that matches the first audio input in the determined screen area, if the determined screen area contains more than one object that matches the first audio input.

47. The computer readable medium of claim 46, wherein using a second input includes:

marking each object matching the first audio input;

receiving a second audio input relating to the selection of one of the marked objects; and

selecting the marked object that best matches the second audio input.

48. The computer readable medium of claim 47, wherein the method further comprises removing the markers.

49. The computer readable medium of claim 47, wherein marking the objects includes marking the objects with icons.

50. The computer readable medium of claim 49, wherein marking the objects with the icons includes marking the objects with semi-transparent icons.

51. The computer readable medium of claim 50, wherein marking the objects with semi-transparent icons includes marking the objects with semi-transparent icons that overlap the objects.

52. The computer readable medium of claim 51, wherein marking the objects with semi-transparent icons that overlap the objects includes marking the objects with semi-transparent icons that have numbered labels.

53. The computer readable medium of claim 46, wherein assigning priorities to the screen areas includes assigning priorities to the screen areas based on usage.

54. The computer readable medium of claim 46, wherein assigning priorities to the screen areas includes assigning adjustable priorities to the screen areas.

55. The computer readable medium of claim 46, wherein the method further comprises indicating the one of the screen areas having the highest priority.

56. The computer readable medium of claim 55, wherein indicating the highest priority screen area includes indicating the highest priority screen area with a visual output.

57. The computer readable medium of claim 56, wherein indicating the highest priority screen area with visual output includes indicating the highest priority screen area with a visual output that includes highlighting.

58. The computer readable medium of claim 46, wherein the method further comprises confirming the receipt of the first audio input.

59. The computer readable medium of claim 58, wherein confirming the receipt of the first audio input includes confirming the receipt of the first audio input with an audio output.

60. A computer readable medium containing instructions for controlling a computer system to perform a method for providing speech control to a graphical user interface (GUI) containing objects to be selected, the method comprising:

receiving a first audio input relating to the selection of one of the objects in the GUI;

determining the one of the screen areas having the highest priority and including a first object matching the first audio input;

selecting the first object, if the determined screen area only contains one object matching the first audio input; and

using a second input to select one of the objects that matches the first audio input in the determined screen area, if the determined screen area contains more than one object that matches the first audio input.

61. An apparatus for providing speech control to a graphical user interface (GUI) containing objects to be selected the apparatus comprising:

means for dividing the GUI into a Plurality of screen areas;

means for assigning priorities to the screen areas;

means for receiving a first audio input relating to the selection of one of the objects in the GUI;

means for determining the one of the screen areas that has the highest priority and includes a first object matching the first audio input;

means for selecting the first object, if the determined screen area only contains one object matching the first audio input; and

means for using a second input to select one of the objects that matches the first audio input in the determined screen area, if the determined screen area contains more than one object that matches the first audio input.

62. The apparatus of claim 61, wherein the means for using a second input includes:

means for marking each object matching the first audio input;

means for receiving a second audio input relating to the selection of one of the marked objects; and

means for selecting the marked object that best matches the second audio input.

63. The apparatus of claim 62, further comprising means for removing the markers.

64. The apparatus of claim 62, wherein the markers include icons.

65. The apparatus of claim 64, wherein the icons are semi-transparent.

66. The apparatus of claim 65, wherein the icons overlap the objects matching the audio input.

67. The apparatus of claim 66, wherein the icons include numbered labels.

68. The apparatus of claim 61, wherein the priorities of the screen areas are based on usage.

69. The apparatus of claim 61, wherein the priorities of the screen areas are adjustable.

70. The apparatus of claim 61, further comprising means for indicating the one of the screen areas having the highest priority.

71. The apparatus of claim 70, wherein the means for indicating the highest priority screen area includes means for indicating the highest priority screen area with a visual output.

72. The apparatus of claim 71, wherein the means for indicating the highest priority screen area with visual output includes means for indicating the highest priority screen area with a visual output that includes highlighting.

73. The apparatus of claim 61, further comprising means for confirming the receipt of the first audio input.

74. The apparatus of claim 73, wherein confirming the receipt of the first audio input includes audio output.

75. An apparatus for providing speech control to a graphical user interface (GUI) containing objects to be selected, the apparatus comprising:

means for receiving a first audio input relating to the selection of one of the objects in the GUI;

means for determining the one of the screen areas that has the highest priority and includes a first object matching the first audio input;

means for selecting the first object, if the determined screen area only contains one object matching the first audio input; and

means for using a second input to select one of the objects that matches the first audio input in the determined screen area, if the determined screen area contains more than one object that matches the first audio input.