LEARNING INTENDED USER ACTIONS
First Claim
1. A system, comprisinga microphone and camera for receiving user utterances indicative of user commands and associated user gestures for the user utterances;
- a hardware-based recognizer for parsing sample utterances and the user utterances into verb parts and noun parts, and recognizing the user utterances and the associated user gestures by comparing the verb parts and the noun parts from the user utterances individually and as pairs to the verb parts and the noun parts of the sample utterances; and
a user command selective execution device for selectively performing a given one of the user commands responsive to a recognition result.
1 Assignment
0 Petitions
Accused Products
Abstract
A method and system are provided. The method includes receiving, by a microphone and camera, user utterances indicative of user commands and associated user gestures for the user utterances. The method further includes parsing, by a hardware-based recognizer, sample utterances and the user utterances into verb parts and noun parts. The method also includes recognizing, by a hardware-based recognizer, the user utterances and the associated user gestures based on the sample utterances and descriptions of associated supporting gestures for the sample utterances. The recognizing step includes comparing the verb parts and the noun parts from the user utterances individually and as pairs to the verb parts and the noun parts of the sample utterances. The method additionally includes selectively performing a given one of the user commands responsive to a recognition result.
-
Citations
20 Claims
-
1. A system, comprising
a microphone and camera for receiving user utterances indicative of user commands and associated user gestures for the user utterances; -
a hardware-based recognizer for parsing sample utterances and the user utterances into verb parts and noun parts, and recognizing the user utterances and the associated user gestures by comparing the verb parts and the noun parts from the user utterances individually and as pairs to the verb parts and the noun parts of the sample utterances; and a user command selective execution device for selectively performing a given one of the user commands responsive to a recognition result. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8)
-
-
9. A computer program product for recognizing intended user actions, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method comprising:
-
receiving, by a microphone and camera, user utterances indicative of user commands and associated user gestures for the user utterances; parsing sample utterances and the user utterances into verb parts and noun parts, and recognizing the user utterances and the associated user gestures by comparing the verb parts and the noun parts from the user utterances individually and as pairs to the verb parts and the noun parts of the sample utterances; and selectively performing a given one of the user commands responsive to a recognition result. - View Dependent Claims (10, 11, 12, 13, 15, 16, 17, 18)
-
-
14. The computer program product of claim 22, wherein the method further comprises generating respective error values for at least one of the noun, the verb, the gesture, and a combination thereof including at least the gesture, responsive to at least one of a number of user accepted examples and a number of user rejected examples involving the gesture and at least one of the noun and the verb for a particular one of the user commands.
-
19. A system, comprising:
-
a processor operatively coupled to a computer-readable storage medium, the processor being configured for; receiving user utterances indicative of user commands and associated user gestures for the user utterances; parsing sample utterances and the user utterances into verb parts and noun parts, and recognizing the associated user gestures based on descriptions of associated supporting gestures for the sample utterance by sequentially comparing the verb parts and the noun parts from the user utterances individually and as pairs to the verb parts and the noun parts of the sample utterances; and selectively performing a given one of the user commands responsive to a recognition result. - View Dependent Claims (20)
-
Specification