Gesture-based input interface system with shadow detection
First Claim
1. A method of providing gesture-based user control of an application running on a computer, he method comprising the steps of:
- receiving at least one image signal; and
processing the at least one image signal to determine if the image contains one of a plurality of designated user gestures, each of the gestures being translatable to a particular control signal for controlling the application, the processing step for a given one of the designated user gestures determining position and orientation information for a pointing finger of a hand of the user and its corresponding shadow, the position and orientation information for the pointing finger and its shadow being utilized to generate a three-dimensional pose estimate for the pointing figure in the given gesture.
9 Assignments
0 Petitions
Accused Products
Abstract
An input interface system provides gesture-based user control of an application running on a computer by classification of user gestures in image signals. A given one of the image signals is processed to determine if it contains one of a number of designated user gestures, e.g., a point gesture, a reach gesture and a click gesture, each of the gestures being translatable to a particular control signal for controlling the application. If the image signal is determined to contain a point gesture, further processing is performed to determine position and orientation information for a pointing finger of a hand of the user and its corresponding shadow. The position and orientation information for the pointing finger and its shadow are then utilized to generate a three-dimensional pose estimate for the pointing figure in the point gesture. For example, the three-dimensional pose estimate may be in the form of a set of five parameters (X, Y, Z, α, ε), where (X, Y, Z) denotes the position of a tip of the pointing finger in three-dimensional space, and (α, ε) denotes the respective azimuth and elevation angles of an axis of the pointing finger. The point gesture can thus be used to provide user control in virtual flight simulators, graphical editors, video games and other applications.
443 Citations
14 Claims
-
1. A method of providing gesture-based user control of an application running on a computer, he method comprising the steps of:
-
receiving at least one image signal; and
processing the at least one image signal to determine if the image contains one of a plurality of designated user gestures, each of the gestures being translatable to a particular control signal for controlling the application, the processing step for a given one of the designated user gestures determining position and orientation information for a pointing finger of a hand of the user and its corresponding shadow, the position and orientation information for the pointing finger and its shadow being utilized to generate a three-dimensional pose estimate for the pointing figure in the given gesture. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11)
-
-
12. An apparatus for providing gesture-based user control of an application running on a computer, the apparatus comprising:
a processing device associated with the computer and operative to process at least one image signal to determine if the image contains one of a plurality of designated user gestures, each of the gestures being translatable to a particular control signal for controlling the application, the processing step for a given one of the designated user gestures determining position and orientation information for a pointing finger of a hand of the user and its corresponding shadow, the position and orientation information for the pointing finger and its shadow being utilized to generate a three-dimensional pose estimate for the pointing figure in the given gesture.
-
13. An input interface system for providing gesture-based user control of an application, the system comprising:
-
at least one camera;
a light source arranged so as to create a shadow of a user hand within a field of view of the camera; and
a computer coupled to the camera and operative to process at least one image signal received from the camera to determine if the image contains one of a plurality of designated user gestures, each of the gestures being translatable to a particular control signal for controlling the application, the processing step for a given one of the designated user gestures determining position and orientation information for a pointing finger of a hand of the user and its corresponding shadow, the position and orientation information for the pointing finger and its shadow being utilized to generate a three-dimensional pose estimate for the pointing figure in the given gesture.
-
-
14. An article of manufacture containing one or more software programs which, when executed in a processing device, cause the processing device to perform the step of processing at least ode image signal to determine if the image contains one of a plurality of designated user gestures, each of the gestures being translatable to a particular control signal for controlling the application, the processing step for a given one of the designated user gestures determining position and orientation information for a pointing finger of a hand of the user and its corresponding shadow, the position and orientation information for the pointing finger and its shadow being utilized to generate a three-dimensional pose estimate for the pointing figure in the given gesture.
Specification