Gesture-controlled interfaces for self-service machines and other applications

• US 7,668,340 B2
• Filed: 12/02/2008
• Issued: 02/23/2010
• Est. Priority Date: 08/10/1998
• Status: Expired due to Term

• Current Assignee: Jolly Seven, Series 70 of Allied Security Trust I (Allied Security Trust)
• Original Assignee: Cybernet Systems Corporation
• Inventors: Cohen, Charles J., Beach, Glenn J., Cavell, Brook, Foulk, Eugene, Jacobus, Charles J., Obermark, Jay, Paul, George V.
• Primary Examiner(s): Tabatabai, Abolfazl
• Application Number:
  US12/326,540
• Publication Number:
  US 20090074248A1
• Time in Patent Office: 448 Days
• Field of Search: 382/100, 382/103, 382/107, 382/153, 382/170, 382190-192, 382/209, 382/219, 382/276, 382/277, 382/280, 382/281, 382/302, 345/156, 345/473, 345/474, 345/520, 345/650, 345/676, 348/147, 348/169, 348170-172, 701/45, 715/523, 715/700, 715/718, 715/740, 715/863
• US Class Current: 382/103
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

A gesture recognition interface for use in controlling self-service machines and other devices is disclosed. A gesture is defined as motions and kinematic poses generated by humans, animals, or machines. Specific body features are tracked, and static and motion gestures are interpreted. Motion gestures are defined as a family of parametrically delimited oscillatory motions, modeled as a linear-in-parameters dynamic system with added geometric constraints to allow for real-time recognition using a small amount of memory and processing time. A linear least squares method is preferably used to determine the parameters which represent each gesture. Feature position measure is used in conjunction with a bank of predictor bins seeded with the gesture parameters, and the system determines which bin best fits the observed motion. Recognizing static pose gestures is preferably performed by localizing the body/object from the rest of the image, describing that object, and identifying that description. The disclosure details methods for gesture recognition, as well as the overall architecture for using gesture recognition to control of devices, including self-service machines.

Claims

1. A control method, comprising the steps of:

storing, in a memory, information relating to a plurality of gestures, the stored information including a plurality of geometric templates associated with static gestures and a plurality of dynamic models associated with dynamic gestures;

receiving information from an image sensor about the position, orientation or x-y movement of a gesture-making target; and

providing at least one processor to perform the following operations;

a) identify the gesture as a static gesture or no gesture if the x-y movement is below a threshold amount, or identify the gesture as a dynamic gesture if the x-y movement is above the threshold amount;

b) compare the position or orientation information of the gesture-making target to the geometric templates to determine if a particular static gesture is being made or, if the gesture is identified as a dynamic gesture, compare the position, orientation or x-y movement information to the stored dynamic models to determine if a particular dynamic gesture is being made; and

c) output a control signal if a particular static or dynamic gesture is determined to control a computer or machine.

2. The method of claim 1, wherein the target is a human hand.

3. The method of claim 1, further including the step of generating a bounding box around the target.

4. The method of claim 1, wherein the geometric template information relating to a plurality of static gestures includes edge information.

5. The method of claim 1, wherein at least a portion of the information received about the position, orientation or x-y movement of the gesture-making target is derived by imaging the target.

6. The method of claim 1, wherein at least a portion of the information received about the position, orientation or x-y movement of the gesture-making target is delved through an accelerometer.

7. The method of claim 1, wherein at least a portion of the information received about the position, orientation or x-y movement of the gesture-making target is derived through an inertial system.

8. The method of claim 1, wherein at least a portion of the information received about the position, orientation or x-y movement of the gesture-making target is derived through a radio frequency communication.

9. The method of claim 1, wherein at least a portion of the information received about the position, orientation or x-y movement of the gesture-making target is derived through acoustic tracking.

10. The method of claim 1, wherein at least a portion of the information received about the position, orientation or x-y movement of the gesture-making target is derived through a mechanical linkage.

11. The method of claim 1, including the step of:

providing a plurality of predictor bins, each containing a dynamic system model with parameters preset to a specific dynamic gesture.

12. The method of claim 11, including separate predictor bins for movements along the x and y axes.