Method, apparatus and computer program product for recognizing a gesture

• US 8,194,925 B2
• Filed: 07/18/2011
• Issued: 06/05/2012
• Est. Priority Date: 03/16/2009
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Higgins, Robert P.
• Primary Examiner(s): AZARIAN, SEYED H
• Application Number:
  US13/184,758
• Publication Number:
  US 20110274358A1
• Time in Patent Office: 323 Days
• Field of Search: 382/100, 382/103, 382/104, 382/106, 382/107, 382/115, 382/153, 382/154, 382/162, 382/168, 382/173, 382/181, 382189-203, 382/209, 382/224, 382/232, 382/254, 382/274, 382/276, 382286-295, 382/305, 382/312, 703/6, 704/270, 1/1, 701/2
• US Class Current: 382/103
• CPC Class Codes: G05D 1/0083 to help an aircraft pilot in the rolling phase
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

A method, apparatus and computer program product are provided for recognizing a gesture in which one or more relationships are determined between a plurality of body parts and the gesture is then determined based upon these relationships. Each relationship may be determined by determining an angle associated with at least one joint, determining one or more states of a body part based upon the angle associated with at least one joint, and determining a probability of a body part being in each respective state. The gesture may thereafter be determined based upon the one or more states and the probability associated with each state of the body part. Directions may be provided, such as to an unmanned vehicle, based upon the gesture to, for example, control its taxiing and parking operations.

Claims

1. A method for recognizing a gesture comprising:

determining one or more relationships between a plurality of body parts, wherein determining each relationship comprises;

determining an angle associated with at least one joint; and

determining one or more states of a body part based upon the angle associated with at least one joint; and

determining a probability of the body part being in each respective state; and

determining the gesture with a gesture recognition unit based upon the one or more states and the probability associated with each state of the body part, wherein determining the gesture comprises identifying one or more candidate gestures having respective probabilities of being the gesture and reporting that the gesture was not recognized in an instance in which the respective probabilities fail to satisfy a predefined threshold.

2. A method according to claim 1 wherein determining the gesture comprises utilizing a dynamic Bayesian network to determine the gesture.

3. A method according to claim 2 wherein determining the gesture further comprises determining at least one subgesture utilizing another dynamic Bayesian network prior to utilizing the dynamic Bayesian network to determine the gesture based at least in part upon the subgesture.

4. A method according to claim 1 wherein determining the gesture comprises determining a respective probability associated with each of a plurality of different gestures.

5. A method according to claim 1 wherein determining one or more relationships between a plurality of body parts comprises:

determining the angle associated with a first joint;

determining one or more states of a first body part based upon the angle associated with the first joint;

determining a probability of the first body part being in each respective state;

determining the angle associated with a second joint;

determining one or more states of a second body part based upon the angle associated with the second joint; and

determining a probability of the second body part being in each respective state.

6. A method according to claim 5 wherein the first joint is a shoulder with the associated angle defined by the elbow, shoulder and torso and the first body part is an upper arm, and wherein the second joint is an elbow with the associated angle defined by the shoulder, elbow and wrist and the second body part is a forearm.

7. A method according to claim 1 further comprising providing directions to an unmanned vehicle based upon the gesture.

8. An apparatus for recognizing a gesture comprising:

a processor configured to determine one or more relationships between a plurality of body parts, wherein the processor is configured to determine each relationship by;

determining an angle associated with at least one joint; and

determining one or more states of a body part based upon the angle associated with at least one joint; and

determining a probability of the body part being in each respective state, and

wherein the processor is configured to determine the gesture based upon the one or more states and the probability associated with each state of the body part, and wherein the processor is configured to determine the gesture by identifying one or more candidate gestures having respective probabilities of being the gesture and reporting that the gesture was not recognized in an instance in which the respective probabilities fail to satisfy a predefined threshold.

9. An apparatus according to claim 8 wherein the processor is configured to determine the gesture by utilizing a dynamic Bayesian network.

10. An apparatus according to claim 9 wherein the processor is configured to determine the gesture by determining at least one subgesture utilizing another dynamic Bayesian network prior to utilizing the dynamic Bayesian network to determine the gesture based at least in part upon the subgesture.

11. An apparatus according to claim 8 wherein the processor is configured to determine the gesture by determining a respective probability associated with each of a plurality of different gestures.

12. An apparatus according to claim 8 wherein the processor is configured to determine one or more relationships between a plurality of body parts by:

determining the angle associated with a first joint;

determining one or more states of a first body part based upon the angle associated with the first joint;

determining a probability of the first body part being in each respective state;

determining the angle associated with a second joint;

determining one or more states of a second body part based upon the angle associated with the second joint; and

determining a probability of the second body part being in each respective state.

13. An apparatus according to claim 12 wherein the first joint is a shoulder with the associated angle defined by the elbow, shoulder and torso and the first body part is an upper arm, and wherein the second joint is an elbow with the associated angle defined by the shoulder, elbow and wrist and the second body part is a forearm.

14. An apparatus according to claim 8 wherein the processor is further configured to provide directions to an unmanned vehicle based upon the gesture.

15. A computer program product for recognizing a gesture, wherein the computer program product comprises at least one computer-readable storage medium having computer-executable program instructions stored therein, the computer-executable program instructions comprising:

program instructions configured to determine one or more relationships between a plurality of body parts, wherein the program instructions configured to determine each relationship comprise;

program instructions configured to determine an angle associated with at least one joint;

program instructions configured to determine one or more states of a body part based upon the angle associated with at least one joint; and

program instructions configured to determine a probability of the body part being in each respective state; and

program instructions configured to determine the gesture based upon one or more states and the probability associated with each state of the body part, wherein the program instructions configured to determine the gesture comprise program instructions configured to identify one or more candidate gestures having respective probabilities of being the gesture and program instructions configured to report that the gesture was not recognized in an instance in which the respective probabilities fail to satisfy a predefined threshold.

16. A computer program product according to claim 15 wherein the program instructions configured to determine the gesture comprise program instructions configured to utilize a dynamic Bayesian network to determine the gesture.

17. A computer program product according to claim 16 wherein the program instructions configured to determine the gesture comprise program instructions configured to determine at least one subgesture utilizing another dynamic Bayesian network prior to utilizing the dynamic Bayesian network to determine the gesture based at least in part upon the subgesture.

18. A computer program product according to claim 15 wherein the program instructions configured to determine the gesture comprise program instructions configured to determine a respective probability associated with each of a plurality of different gestures.

19. A computer program product according to claim 15 wherein the program instructions configured to determine one or more relationships between a plurality of body parts comprise:

program instructions configured to determine the angle associated with a first joint;

program instructions configured to determine one or more states of a first body part based upon the angle associated with the first joint;

program instructions configured to determine the probability of the first body part being in each respective state;

program instructions configured to determine the angle associated with a second joint;

program instructions configured to determine one or more states of a second body part based upon the angle associated with the second joint; and

program instructions configured to determine the probability of the second body part being in each respective state.

20. A computer program product according to claim 19 wherein the first joint is a shoulder with the associated angle defined by the elbow, shoulder and torso and the first body part is an upper arm, and wherein the second joint is an elbow with the associated angle defined by the shoulder, elbow and wrist and the second body part is a forearm.