System and method for menu-driven voice control of characters in a game environment

• US 7,085,722 B2
• Filed: 05/14/2001
• Issued: 08/01/2006
• Est. Priority Date: 05/14/2001
• Status: Expired due to Term

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment America (Sony Group Corp.)
• Inventors: Luisi, Seth C. H.
• Primary Examiner(s): McFadden, Susan
• Assistant Examiner(s): Vo, Huyen X.
• Application Number:
  US09/859,034
• Publication Number:
  US 20020169617A1
• Time in Patent Office: 1,905 Days
• Field of Search: 704/231, 704/246, 704/270, 704/275, 704/272, 704/251, 704/270.1, 463 1- 8, 463/29, 463/30, 463/36, 463/39, 463 49- 53, 345/156
• US Class Current: 704/275
• CPC Class Codes: A63F 13/10 Control of the course of the game, e.g. start, progess, end
• CPC Class Codes: A63F 13/215 comprising means for detecting acoustic signals, e.g. using a microphone
• CPC Class Codes: A63F 13/424 involving acoustic input signals, e.g. by using the results of pitch or rhythm extraction or voice recognition
• CPC Class Codes: A63F 13/45 Controlling the progress of the video game
• CPC Class Codes: A63F 13/533 for prompting the player, e.g. by displaying a game menu
• CPC Class Codes: A63F 2300/1081 Input via voice recognition
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/228 of application context

Abstract

In a gaming system, a user controls actions of characters in the game environment using speech commands. In a learning mode, available speech commands are displayed in a command menu on a display device. In a non-learning mode, the available speech commands are not displayed. A speaker-independent context-sensitive speech recognition module contains a vocabulary of available speech commands. Use of speech commands is combined with input from a controller device to control actions of a character or characters in the game environment.

Claims

1. A system for voice control of characters in a visually dynamic and interactive gaming environment, comprising:

a microphone operable to receive a plurality of spoken commands from a user, each of the plurality of spoken commands corresponding to a command selection for each level of a plurality of command levels;

a speech recognition module stored in a memory, the speech recognition module configured to recognize the plurality of spoken commands; and

software stored in the memory, the software configured to respond to a complete and uninterrupted command recognized from a vocabulary of commands to control visual actions of at least one character in the visually dynamic and interactive gaming environment, wherein the complete and uninterrupted command is a combination of the plurality of spoken commands from at least two different levels, and wherein the software may be configured to optionally display a menu of available command selections for each level of a plurality of command levels.

2. The system of claim 1, wherein the software displays the menu of available command selections on a display device.

3. The system of claim 1, wherein the software displays a level 1 menu of available level 1 command selections on a display device in response to a signal from a controller device.

4. The system of claim 3, wherein the software displays a level 2 menu of available level 2 command selections on the display device in response to the speech recognition module recognizing the level 1 command selection spoken by the user.

5. The system of claim 4, wherein the software controls the visual actions of the at least one character in response to the speech recognition module recognizing from the vocabulary of commands a combination of the level 1 command selection and the level 2 command selection spoken by the user as the complete and uninterrupted command.

6. The system of claim 4, wherein the software displays a level 3 menu of available level 3 command selections on the display device in response to the speech recognition module recognizing the level 2 command selection spoken by the user.

7. The system of claim 6, wherein the software controls the visual actions of the at least one character in response to the speech recognition module recognizing from the vocabulary of commands a combination of the level 1 command selection the level 2 command selection, and the level 3 command selection spoken by the user as the complete and uninterrupted command.

8. The system of claim 1, further comprising a controller device for inputting commands to the software to control visual actions of a second character.

9. The system of claim 8, wherein the user controls the visual actions of the second character using the controller device and controls the visual actions of the at least one character using the spoken commands.

10. The system of claim 1, wherein the complete and uninterrupted command comprises at least a first word and a second word, the first word corresponding to a first spoken command associated with a selection for a first command level of the plurality of command levels, and the second word corresponding to a second spoken command associated with a selection for a second command level of the plurality of command levels.

11. The system of claim 1, wherein the software is further configured to generate an audible confirmation of recognition of the complete and uninterrupted command once the complete and uninterrupted command is recognized from the vocabulary of commands.

12. A method for controlling visual actions of at least one character in a visually dynamic and interactive gaming environment, comprising:

displaying a plurality of menus of command selections to a user, each menu of the plurality of menus of command selections corresponding to a command level of a plurality of command levels;

recognizing a plurality of utterances, each utterance of the plurality of utterances corresponding to a command selection from each menu of the plurality of menus of command selections, the plurality of utterances collectively representing a complete and uninterrupted command; and

causing the at least one character to perform a visual action corresponding to the complete and uninterrupted command if the plurality of utterances correspond to a complete and uninterrupted command recognized from a vocabulary of commands, wherein the complete and uninterrupted command comprises a combination of command selections from at least two menus of the plurality of menus of command selections.

13. The method of claim 12, wherein one menu of the plurality of menus of command selections comprises a menu of available characters.

14. The method of claim 12, wherein at least one menu of the plurality of menus of command selections comprises a menu of available visual actions for the at least one character, where command selections in the menu of available actions depends upon a state of the visually dynamic and interactive gaming environment.

15. The method of claim 12, wherein recognizing the plurality of utterances includes evaluating a context of the plurality of utterances.

16. The method of claim 12, wherein the complete and uninterrupted command comprises at least two words.

17. The method of claim 12, wherein the complete and uninterrupted command is a sentence-type command.

18. The method of claim 12, further comprising emitting an audible confirmation of recognition of the complete and uninterrupted command once the complete and uninterrupted command is recognized from the vocabulary of commands.

19. A computer-readable medium comprising instructions for controlling visual actions of at least one character in a visually dynamic and interactive gaming environment, the instructions comprising:

displaying a plurality of menus of command selections to a user, each menu of the plurality of menus of command selections corresponding to a command level of a plurality of command levels;

recognizing a plurality of utterances, each utterance of the plurality of utterances corresponding to a command selection from each menu of the plurality of menus of command selections, the plurality of utterances collectively representing a complete and uninterrupted command; and

causing the at least one character to perform a visual action corresponding to the complete and uninterrupted command if the plurality of utterances correspond to a complete and uninterrupted command recognized from a vocabulary of commands, wherein the complete and uninterrupted command comprises a combination of command selections from at least two menus of the plurality of menus of command selections.

20. The computer-readable medium of claim 19, wherein one menu of the plurality of menus of command selections comprises a menu of available characters.

21. The computer readable-medium of claim 19, wherein at least one menu of the plurality of menus of command selections comprises a menu of available visual actions for the at least one character, where command selections in the menu of available actions depends upon a state of the visually dynamic and interactive gaming environment.

22. The computer-readable medium of claim 19, wherein recognizing the plurality of utterances includes evaluating a context of the plurality of utterances.

23. The computer-readable medium of claim 19, wherein the complete and uninterrupted command comprises at least two words.

24. The computer-readable medium of claim 19, wherein the complete and uninterrupted command is a sentence-type command.

25. The computer-readable medium of claim 19, further comprising instructions for causing the emission of an audible confirmation of recognition of the complete and uninterrupted command once the complete and uninterrupted command is recognized from the vocabulary of commands.

26. A system for controlling visual actions of at least one character in a visually dynamic and interactive gaming environment, comprising:

means for displaying a plurality of menus of command selections to a user, each menu of the plurality of menus of command selections corresponding to a command level of a plurality of command levels;

means for recognizing a plurality of utterances, each utterance of the plurality of utterances corresponding to a command selection from each menu of the plurality of menus of command selections, the plurality of utterances collectively representing a complete and uninterrupted command; and

means for causing the at least one character to perform a visual action corresponding to the complete and uninterrupted command if the plurality of utterances correspond to a complete and uninterrupted command recognized from a vocabulary of commands, wherein the complete and uninterrupted command comprises a combination of command selections from at least two menus of the plurality of menus of command selections.

27. The system of claim 26, wherein one menu of the plurality of menus of command selections comprises a menu of available characters.

28. The system of claim 26, wherein at least one menu of the plurality of menus of command selections comprises a menu of available visual actions for the at least one character, where command selection in the menu of available actions depends upon a state of the visually dynamic and interactive gaming environment.

29. The system of claim 26, wherein the means for recognizing the plurality of utterances evaluates a context of the plurality of utterances.

30. The system of claim 26, wherein the complete and uninterrupted command comprises at least two words.

31. The system of claim 26, wherein the complete and uninterrupted command is a sentence-type command.

32. The system of claim 26, wherein the means for recognizing the plurality of utterances is a speaker-independent context-sensitive speech recognizer.

33. The system of claim 26, wherein the plurality of menus of command selections is displayed to the user as a branching menu.

34. The system of claim 26, further comprising means for emitting an audible confirmation of recognition of the complete and uninterrupted command once the complete and uninterrupted command is recognized from the vocabulary of commands.