Voice-controlled computer simulateously displaying application menu and list of available commands

• US 5,890,122 A
• Filed: 02/08/1993
• Issued: 03/30/1999
• Est. Priority Date: 02/08/1993
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Hysom, Shannon Scott, Van Kleeck, Michael Hinkley
• Primary Examiner(s): Hudspeth, David R.
• Assistant Examiner(s): Smits, Talivaldis Ivars
• Application Number:
  US08/014,706
• Time in Patent Office: 2,241 Days
• Field of Search: 381/43, 381/44, 381/48, 395/2.4, 395/2.44, 395/2.6, 395/2.84, 704/231, 704/235, 704/251, 704/275
• US Class Current: 704/275
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/228 of application context

Abstract

A method and system in a computer system for communicating instructions to an application program in response to spoken commands received by a spoken command input facility is provided. In a preferred embodiment of the present invention, a list of available commands is displayed under the control of the spoken command input facility. Each available command has an instruction associated with it that causes the application program to take some action. The spoken command input facility then determines whether a command spoken by a user is an available command. The spoken command input facility then communicates the instruction associated with the determined available command to the application program. If the spoken command is an available command, the spoken command input facility then revises the displayed list of available commands.

Claims

1. A method in a computer system for indicating to a user a list of available voice commands that are currently recognizable by the computer system, the computer system having an application program that performs tasks in response to receiving instructions to perform the tasks, the method comprising:

providing a plurality of lists of available voice commands, each voice command in a list being associated with an instruction for a task and being associated with an indication of a list of available voice commands to display next;

displaying menu items of a menu of the application program;

displaying, separately from the displayed menu items, one of the lists of available voice commands, the displayed list having an available voice command that corresponds to a displayed menu item and having an available voice command that does not correspond to a displayed menu item so that available voice commands other than those corresponding to the displayed menu items are displayed;

while displaying the menu items and one of the lists of available voice commands, receiving speech corresponding to one of the displayed available voice commands; and

in response to receiving the speech,

sending the instruction associated with the available voice command corresponding to the received speech to the application program so that the application program performs a task in response to receiving the sent instruction; and

displaying the list of available voice commands indicated to be displayed next that is associated with the available voice command corresponding to the received speech

whereby the menu items of the menu of the application program and the list of available voice commands are displayed simultaneously.

2. The method of claim 1 wherein an available voice command that corresponds to a menu item is a different word from the menu item.

3. The method of claim 1 including providing a visual indication that the received speech has been recognized as one of the available voice commands in the displayed list of available voice commands.

4. A method in a computer system for executing instructions in response to spoken commands, the method comprising:

displaying currently available menu items;

displaying, separately from the displayed menu items, a list of available voice commands so that the currently available menu items and the list of available voice commands are displayed simultaneously, an instruction being associated with each available voice command;

while the currently available menu items and the list of available voice commands are displayed, receiving speech spoken by a user;

determining whether the received speech corresponds to an available voice command; and

when it is determined that the received speech corresponds to an available voice command,

executing the instruction associated with the determined available voice command; and

revising the displayed list of available voice commands to reflect those commands that are available after execution of the instruction associated with the available voice command corresponding to the received speech.

5. The method of claim 4 wherein the determining includes subjecting the spoken command to a voice recognition process that indicates that the spoken command is an available voice command when recognized.

6. The method of claim 4, further including indicating that a command has been spoken.

7. The method of claim 4, further including, when the spoken command is determined to be an available voice command, indicating that the spoken voice command is an available voice command.

8. The method of claim 7 wherein the indicating that the spoken command is an available voice command displays the determined available voice command.

9. The method of claim 4, further including, when the spoken command is determined to not be an available voice command, indicating that the spoken command is not an available voice command.

10. In a computer system having an application program capable of performing a task, a method for performing the task under the control of the application program in response to receiving voice data, the method comprising:

under the control of the application program,

displaying in an application program window one or more menus each comprised of menu items corresponding to a task of the application program;

under the control of a voice input facility that is separate from the application program,

displaying a plurality of available voice commands in a voice input facility window, one of the voice commands being associated with a menu item such that the available voice commands and the one or more menus are displayed simultaneously;

while the one or more menus are displayed simultaneously with the plurality of available voice commands, receiving voice data;

recognizing the voice data as a voice command among the displayed plurality of available voice commands;

submitting a request to the application program to perform the task of the menu item associated with the voice command;

under the control of the application program,

in response to the request, performing the functionality of the menu item associated with the voice command; and

under the control of the voice input facility,

revising the plurality of available voice commands displayed in the voice input facility window to reflect voice commands available after the application program has begun performing the task.

11. An apparatus in a computer system for executing instructions in response to spoken commands, the apparatus comprising:

means for displaying currently available menu items;

means for displaying, simultaneously with the displayed menu item and in a command window separate from the menu items, a list of available voice commands, an instruction being associated with each available voice command;

means for determining whether a spoken command is an available voice command, the spoken command being spoken while the currently available menu items and the list of available voice commands are displayed simultaneously;

means for executing the instruction associated with the determined available voice command; and

means for revising the list of available voice commands displayed in the command window to reflect those commands that are available after the instruction is executed.

12. The apparatus of claim 11 wherein the determining means subjects the spoken command to a voice recognition process that indicates that the spoken command is an available voice command when recognized.

13. The apparatus of claim 11, further including means for indicating that a command has been spoken.

14. The apparatus of claim 11, further including means for indicating that the spoken command is an available voice command when the spoken command is determined to be an available voice command.

15. The apparatus of claim 14 wherein the means for indicating that the spoken command is an available voice command displays the determined available voice command.

16. The apparatus of claim 11, further including means for indicating that the spoken command is not an available voice command when the spoken command is determined to not be an available voice command.

17. A computer-readable medium containing computer instructions for causing a computer system for executing instructions in response to spoken commands, the method comprising:

displaying currently available menu items;

displaying, separately from the displayed menu items, a list of available voice commands so that the currently available menu items and the list of available voice commands are displayed simultaneously, an instruction being associated with each available voice command;

while the currently available menu items and the list of available voice commands are displayed simultaneously, receiving speech spoken by a user;

determining whether the received speech corresponds to an available voice command; and

when it is determined that the received speech corresponds to an available voice command, executing the instruction associated with the determined available voice command.

18. The computer-readable medium of claim 17 wherein the determining includes subjecting the spoken command to a voice recognition process that indicates that the spoken command is an available voice command wherein recognized.

19. The computer-readable medium of claim 17, further including indicating that a command has been spoken.

20. The computer-readable medium of claim 17, further including, when the spoken command is determined to be an available voice command, indicating that the spoken voice command is an available voice command.

21. The computer-readable medium of claim 20 wherein the indicating that the spoken command is an available voice command displays the determined available voice command.

22. The computer-readable medium of claim 17, further including, when the spoken command is determined to not be an available voice command, indicating that the spoken command is not an available voice command.

23. A method in a computer system for executing instructions in response to spoken commands, the method comprising the steps of:

displaying currently available menu items;

displaying, in a command window distinct from the menu items, a list of available commands, each available command having an instruction associated with it;

determining whether a spoken command is an available command;

executing the instruction associated with the determined available command; and

revising the list of available commands displayed in the command window to reflect those commands that are available after execution of the instruction associated with the determined available command.

24. The method of claim 23 wherein the determining step includes subjecting the spoken command to a voice recognition process that indicates that the spoken command is an available command when recognized.

25. The method of claim 23, further including the step of indicating that a command has been spoken.

26. The method of claim 23, further including the step of, when the spoken command is determined to be an available command, indicating that the spoken command is an available command.

27. The method of claim 26 wherein the step of indicating that the spoken command is an available command displays the determined available command.

28. The method of claim 23, further including the step of, when the spoken command is determined to not be an available command, indicating that the spoken command is not an available command.