GESTURE-CONTROLLED INTERFACES FOR SELF-SERVICE MACHINES AND OTHER APPLICATIONS

• US 20030138130A1
• Filed: 08/10/1999
• Published: 07/24/2003
• Est. Priority Date: 08/10/1998
• Status: Active Grant

• Current Assignee: Jolly Seven, Series 70 of Allied Security Trust I (Allied Security Trust)
• Original Assignee: Cybernet Systems Corporation
• Inventors: COHEN, CHARLES J., JACOBUS, CHARLES J., OBERMARK, JAY, PAUL, GEORGE, FOULK, GENE, CAVELL, BROOK, BEACH, GLENN
• Granted Patent: US 6,681,031 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: G07F 9/023 Arrangements for display, data presentation or advertising

Abstract

A gesture recognition interface for use in controlling self-service machines and other devices is disclosed. A gesture is defined as motions and kinematic poses generated by humans, animals, or machines. Specific body features are tracked, and static and motion gestures are interpreted. Motion gestures are defined as a family of parametrically delimited oscillatory motions, modeled as a linear-in-parameters dynamic system with added geometric constraints to allow for real-time recognition using a small amount of memory and processing time. A linear least squares method is preferably used to determine the parameters which represent each gesture. Feature position measure is used in conjunction with a bank of predictor bins seeded with the gesture parameters, and the system determines which bin best fits the observed motion. Recognizing static pose gestures is preferably performed by localizing the body/object from the rest of the image, describing that object, and identifying that description. The disclosure details methods for gesture recognition, as well as the overall architecture for using gesture recognition to control of devices, including self-service machines.

Claims

1. A method of gesture recognition, comprising the steps of:

imaging a gesture-making target;

deriving the start position of the target, the end position of the target, and the velocity between the start and end positions;

comparing the velocity of the target to a threshold value; and

identifying the gesture as a static gesture if the velocity is below the threshold value, otherwise,

identifying the gesture as a dynamic gesture.

2. The method of claim 1, wherein the target is a human hand.

3. The method of claim 2, further including the step of generating a bounding box around the hand.

4. The method of claim 1, further including the step of using an operator to find the edges of the target.

5. The method of claim 1, further including the steps of:

receiving a file of recognized gestures along with their vector descriptions; and

comparing static gestures to the vector descriptions.

6. The method of claim 1, further including the step of treating a dynamic gesture as one or more one-dimensional oscillations.

7. The method of claim 6, further including the step of treating a circular motion as a combination of repeating motions in two dimensions having the same magnitude and frequency of oscillation.

8. The method of claim 6, further including the step of deriving complex dynamic gestures by varying phase relationships.

9. The method of claim 6, further including the step of deriving a multi-gesture lexicon based upon clockwise and counter-clockwise large and small circles and one-dimensional lines.

10. The method of claim 6, further including the step of comparing to the next position and velocity of each gesture to one or more predictor bins to determine a gesture'"'"'s future position and velocity.

11. The method of claim 10, further including the use of a linear-with-offset-component model to discriminate among simple dynamic gestures.

12. The method of claim 10, further including the use of a velocity damping model to discriminate among non-circular dynamic gestures.

13. A gesture-controlled interface for self-service machines and other applications, comprising:

a sensor module for visually analyzing a gesture made by a human or machine, and outputting image data including position and velocity information associated with the gesture;

an identification module operative to identify the gesture based upon the image data output by the sensor module; and

a transformation module operative to generate a command based upon the gesture identified by the identification module.

14. The interface of claim 13, further including a system response module operative to apply to the command from the transformation module to a device to be controlled.

15. The interface of claim 14, wherein the device is a virtual-reality simulator.

16. The interface of claim 14, wherein the device is a self-service machine.

17. The interface of claim 14, wherein the device forms part of a robot.

18. The interface of claim 14, wherein the device forms part of a commercial appliance.