Method and system for detecting conscious hand movement patterns and computer-generated visual feedback for facilitating human-computer interaction
First Claim
1. An apparatus for providing a large number of graphical elements to an individual who is interacting with a multimedia terminal, comprising:
- (a) means for capturing a plurality of images from said individual,(b) means for calculating a color segmentation image of said individual in said plurality of images,(c) means for calculating a motion energy image of said individual in said plurality of images,(d) means for calculating a body part location confidence image of said individual by combining said color segmentation image and said motion energy image,wherein combining said color segmentation image and said motion energy image is performed by adding and multiplying the images together, and(e) means for displaying said large number of graphical elements on said multimedia terminal that correspond to the body part location estimates of said individual in the body part location confidence image,whereby said large number of graphical elements provides said individual with information about what the apparatus senses and the certainty with which the apparatus has sensed said body part location estimates,whereby said large number of graphical elements gives said individual an opportunity to have a better understanding about the capabilities of said multimedia terminal to sense said body movements, andwhereby said understanding helps said individual to adapt to the capabilities of said multimedia terminal, leading to an improved interaction experience.
4 Assignments
0 Petitions
Accused Products
Abstract
The present invention is a system and method for detecting and analyzing motion patterns of individuals present at a multimedia computer terminal from a stream of video frames generated by a video camera and the method of providing visual feedback of the extracted information to aid the interaction process between a user and the system. The method allows multiple people to be present in front of the computer terminal and yet allow one active user to make selections on the computer display. Thus the invention can be used as method for contact-free human-computer interaction in a public place, where the computer terminal can be positioned in a variety of configurations including behind a transparent glass window or at a height or location where the user cannot touch the terminal physically.
165 Citations
10 Claims
-
1. An apparatus for providing a large number of graphical elements to an individual who is interacting with a multimedia terminal, comprising:
-
(a) means for capturing a plurality of images from said individual, (b) means for calculating a color segmentation image of said individual in said plurality of images, (c) means for calculating a motion energy image of said individual in said plurality of images, (d) means for calculating a body part location confidence image of said individual by combining said color segmentation image and said motion energy image, wherein combining said color segmentation image and said motion energy image is performed by adding and multiplying the images together, and (e) means for displaying said large number of graphical elements on said multimedia terminal that correspond to the body part location estimates of said individual in the body part location confidence image, whereby said large number of graphical elements provides said individual with information about what the apparatus senses and the certainty with which the apparatus has sensed said body part location estimates, whereby said large number of graphical elements gives said individual an opportunity to have a better understanding about the capabilities of said multimedia terminal to sense said body movements, and whereby said understanding helps said individual to adapt to the capabilities of said multimedia terminal, leading to an improved interaction experience. - View Dependent Claims (2, 3, 4, 5)
-
-
6. A method for providing a large number of graphical elements to an individual who is interacting with a multimedia terminal, comprising the following steps of:
-
(a) capturing a plurality of images from said individual, (b) calculating a color segmentation image of said individual in said plurality of images, (c) calculating a motion energy image of said individual in said plurality of images, (d) calculating a body part location confidence image of said individual by combining said color segmentation image and said motion energy image, wherein combining said color segmentation image and said motion energy image is performed by adding and multiplying the images together, and (e) displaying said large number of graphical elements on said multimedia terminal that correspond to the body part location estimates of said individual in the body part location confidence image, whereby said large number of graphical elements provides said individual with information about the certainty of sensed said body part location estimates, whereby displaying said large number of graphical elements gives said individual an opportunity to have a better understanding about the capabilities of said multimedia terminal, and whereby said understanding helps said individual to adapt to the capabilities of said multimedia terminal, leading to an improved interaction experience. - View Dependent Claims (7, 8, 9, 10)
-
Specification