Behavior recognition system

• US 7,036,094 B1
• Filed: 03/31/2000
• Issued: 04/25/2006
• Est. Priority Date: 08/10/1998
• Status: Expired due to Term

• Current Assignee: Jolly Seven, Series 70 of Allied Security Trust I (Allied Security Trust)
• Original Assignee: Cybernet Systems Corporation
• Inventors: Cavell, Brook, Foulk, Gene, Jacobus, Charles J., Obermark, Jay, Paul, George, Cohen, Charles J., Beach, Glenn
• Primary Examiner(s): Kincaid, Kristine
• Assistant Examiner(s): VU, THANH T
• Application Number:
  US09/540,461
• Time in Patent Office: 2,216 Days
• Field of Search: 345/563, 345/719, 382/168, 382/170, 715/719, 715/740, 715/863, 715/866
• US Class Current: 715/863
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 40/25 Recognition of walking or running movements, e.g. gait recognition
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

A system for recognizing various human and creature motion gaits and behaviors is presented. These behaviors are defined as combinations of “gestures” identified on various parts of a body in motion. For example, the leg gestures generated when a person runs are different than when a person walks. The system described here can identify such differences and categorize these behaviors. Gestures, as previously defined, are motions generated by humans, animals, or machines. Where in the previous patent only one gesture was recognized at a time, in this system, multiple gestures on a body (or bodies) are recognized simultaneously and used in determining behaviors. If multiple bodies are tracked by the system, then overall formations and behaviors (such as military goals) can be determined.

Claims

1. A method of behavior recognition, comprising the steps of:

storing a dynamic motion model composed of a set of differential equations, each differential equation describing a particular dynamic gesture to be recognized of the form;

{dot over (x)}=f(x,θ

)

where x is vector describing position and velocity components, and θ

is a tunable parameter;

capturing the motion to be recognized along with the tunable parameters associated with a gesture-making target;

extracting the position and velocity components of the captured motion; and

identifying the dynamic gesture by determining which differential equation is solved using the extracted components and the tunable parameters;

designating one or more predefined behaviors;

comparing the identified gesture to one of the predefined behaviors; and

in the event of a correlation between the gesture and the particular predefined behavior, determining that the behavior of the target includes the particular gesture.

2. The method of claim 1, wherein the target is a human being.

3. The method of claim 1 wherein the target is a group of people.

4. The method of claim 1, wherein the target is a human hand.

5. The method of claim 1, further including the steps of:

deriving the start position of the target, the end position of the target, and the velocity between the start and end positions;

comparing the velocity of the target to a threshold value; and

identifying the gesture as a static gesture if the velocity is below the threshold value, otherwise,

identifying the gesture as a dynamic gesture.

6. The method of claim 1, wherein the step of analyzing the gesture-making target includes the use of a velocity damping gesture model.

7. The method of claim 1, wherein the step of analyzing the gesture-making target includes imaging the target.

8. The method of claim 7, further including the step of generating a bounding box around the target.

9. The method of claim 7, further including the step of using an operator to find the edges of the target.

10. The method of claim 1, further including the steps of:

receiving a file of recognized gestures along with their vector descriptions; and

comparing the outputs of the gesture recognition modules to the vector descriptions.

11. The method of claim 1, further including the step of treating a gesture as a dynamic gesture comprising one or more one-dimensional oscillations.

12. The method of claim 11, further including the step of treating a circular motion as a combination of repeating motions in two dimensions having the same magnitude and frequency of oscillation.

13. The method of claim 11, further including the step of deriving complex dynamic gestures by varying phase relationships.

14. The method of claim 11, further including the step of deriving a multi-gesture lexicon based upon clockwise and counter-clockwise large and small circles and one-dimensional lines.

15. The method of claim 11, further including the step of comparing to the next position and velocity of each gesture to one or more predictor bins to determine a gesture'"'"'s future position and velocity.

16. The method of claim 15, further including the use of a linear-with-offset-component model to discriminate among simple dynamic gestures.

17. The method of claim 15, further including the use of a velocity damping model to discriminate among non-circular dynamic gestures.

18. The method of claim 1, wherein the target includes a vehicle.

19. The method of claim 1, wherein the target includes a weapon.

20. The method of claim 1, wherein the target forms part of a robot.