Gesture Recognition Apparatus and Method

• US 20100013944A1
• Filed: 07/23/2008
• Published: 01/21/2010
• Est. Priority Date: 10/05/2006
• Status: Active Grant

• Current Assignee: the united states of america as represented by the secretary of the navy
• Original Assignee: the united states of america as represented by the secretary of the navy
• Inventors: Venetsky, Larry, Tieman, Jeffrey W.
• Granted Patent: US 8,005,257 B2
• Time in Patent Office: Days
• US Class Current: 348/222.100
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

A method of identifying a human gesture using a machine includes providing a time sequence of data related to the human gesture; transforming the time sequence of data into waveforms; extracting features from the waveforms; and identifying the human gesture based on the extracted features.

Claims

1. A method of identifying a human gesture, comprising:

providing a time sequence of data related to the human gesture;

transforming the time sequence of data into waveforms;

extracting features from the waveforms; and

identifying the human gesture based on the extracted features.

2. The method of claim 1 wherein the providing step includes providing a time sequence of pixel images of the human gesture using at least one video camera.

3. The method of claim 1 wherein the providing step includes providing the data using accelerometers attached to the human.

4. The method of claim 1 wherein the extracting step includes extracting static features from the waveforms.

5. The method of claim 1 wherein the extracting step includes extracting dynamic features from the waveforms.

6. The method of claim 1 wherein the transforming step includes transforming the data into a right hand waveform and a left hand waveform.

7. The method of claim 6 wherein the extracting step includes extracting hand position, phase and frequency from the right hand waveform and the left hand waveform.

8. The method of claim 1 wherein the transforming step includes transforming the data into waveforms that are normalized to a cranial location.

9. The method of claim 7 wherein the identifying step includes identifying the human gesture by comparing one or more of the hand position, phase and frequency from the right hand waveform and the hand position, phase and frequency from the left hand waveform to at least one rule that describes the human gesture.

10. The method of claim 7 further comprising classifying hand positions into vertical and horizontal ranges.

11. The method of claim 10 wherein the vertical and horizontal ranges are defined in terms of a characteristic length of a human performing the gesture.

12. The method of claim 11 wherein the characteristic length is a distance D from a cranial point to a torso point.

13. The method of claim 12 wherein the vertical ranges comprise five ranges, a VERY HIGH range being more than 0.25 D above the cranial point, a HIGH range being from the cranial point to 0.25 D above the cranial point, a MIDDLE range being from the cranial point to 0.6 D below the cranial point, a LOW range being below 0.6 D but above 0.9 D, and a VERY LOW range being 0.9 D or lower.

14. The method of claim 12 wherein the horizontal ranges comprise three ranges, an INSIDE range being from 0 to less than 0.3 D from a vertical axis of the human, a MIDDLE range being from 0.3 D to 0.7 D from the vertical axis and an OUTSIDE range being greater than 0.7 D from the vertical axis.

15. The method of claim 12 further comprising classifying a hand position as EXTENDED if a distance from the hand position to a point that is 0.375 D vertically below the cranial point is greater than 0.75 D.

16. The method of claim 7 further comprising classifying cyclic hand movement with respect to an axis as IN-PHASE if absolute values of maximum positions of the right and left hands from the origin, measured along the axis, are the same or approximately the same, and absolute values of minimum positions of the right and left hands from the origin, measured along the axis, are the same or approximately the same.

17. The method of claim 10 wherein hand position in the vertical range is determined using a fuzzy logic method.

18. The method of claim 9 wherein the gesture is MOVE AHEAD and the rule is a y axis position of both hands is MIDDLE or HIGH or VERY HIGH;

and x axis and y axis motion of the hands is IN-PHASE.

19. The method of claim 9 wherein the gesture is TURN TO LEFT and the rule is left hand is EXTENDED and y axis position of left hand is MIDDLE and left hand NO MOTION on all axes;

right hand y axis position is MIDDLE or HIGH or VERY HIGH and right hand has MOTION.

20. The method of claim 9 wherein the gesture is TURN TO RIGHT and the rule is right hand is EXTENDED and y axis position of right hand is MIDDLE and right hand NO MOTION on all axes;

left hand y axis position is MIDDLE or HIGH or VERY HIGH and left hand has MOTION.

21. The method of claim 9 wherein the gesture is PIVOT TO LEFT and the rule is left hand y axis position is VERY LOW and EXTENDED and NO MOTION;

right hand y axis position is MIDDLE or HIGH or VERY HIGH and right hand has MOTION.

22. The method of claim 9 wherein the gesture is PIVOT TO RIGHT and the rule is right hand y axis position is VERY LOW and right hand is EXTENDED and right hand has NO MOTION;

left hand y axis position is MIDDLE or HIGH or VERY HIGH and left hand has MOTION.

23. The method of claim 9 wherein the gesture is PASS CONTROL:

HAND OFF LEFT and the rule is both hands y axis position is MIDDLE and both hands x axis position is MIDDLE or OUTSIDE and both hands have NO MOTION and both hands x axis position is negative.

24. The method of claim 9 wherein the gesture is PASS CONTROL:

HAND OFF RIGHT and the rule is both hands y axis position is MIDDLE and both hands x axis position is MIDDLE or OUTSIDE and both hands have NO MOTION and both hands x axis position is positive.

25. The method of claim 9 wherein the gesture is I HAVE COMMAND and the rule is left hand y axis position is VERY HIGH, right hand y axis position is VERY LOW, both hands have NO MOTION, left hand x axis position is approximately equal to right hand x axis position.

26. The method of claim 9 wherein the gesture is SLOW DOWN and the rule is both hands y axis position is MIDDLE or LOW, both hands y axis position is approximately equal, both hands have IN PHASE y axis motion and both hands have NO MOTION on x axis.

27. The method of claim 9 wherein the gesture is STOP and the rule is both hands y axis position is HIGH or VERY HIGH, both hands x axis position is INSIDE, both hands have NO MOTION.

28. The method of claim 9 wherein the gesture is MOVE BACK and the rule is both hands y axis position is VERY LOW or LOW, both hands NO MOTION on x axis, both hands IN PHASE on y axis, both hands y axis position approximately equal, both hands IN PHASE on z axis.

29. The method of claim 9 wherein the gesture is SLOW DOWN ENGINE(S) ON LEFT INDICATED SIDE and the rule is right hand y axis position is VERY LOW, right hand has NO MOTION, left hand y axis position is MIDDLE or VERY LOW, left hand has y axis MOTION.

30. The method of claim 9 wherein the gesture is SLOW DOWN ENGINE(S) ON RIGHT INDICATED SIDE and the rule is left hand y axis position is VERY LOW, left hand has NO MOTION, right hand y axis position is MIDDLE or LOW, right hand has y axis MOTION.

31. The method of claim 9 wherein the gesture is ENGAGE NOSEGEAR STEERING RIGHT and the rule is left hand y axis position is MIDDLE, left hand x axis position is INSIDE, left hand NO MOTION, right hand y axis position is MIDDLE, right hand is EXTENDED, right hand x axis position is OUTSIDE, right hand has NO MOTION.

32. The method of claim 9 wherein the gesture is ENGAGE NOSEGEAR LEFT and the rule is right hand y axis position is MIDDLE, right hand x axis position is INSIDE, right hand has NO MOTION, left hand y axis position is MIDDLE, left hand is EXTENDED, left hand x axis position is OUTSIDE, left hand has NO MOTION.

33. The method of claim 9 wherein the gesture is DISENGAGE NOSEGEAR STEERING RIGHT and the rule is left hand y axis position is MIDDLE, left hand x axis position is INSIDE, left hand has NO MOTION, right hand y axis position is MIDDLE, right hand has y axis MOTION, right hand has NO MOTION on x axis.

34. The method of claim 9 wherein the gesture is DISENGAGE NOSEGEAR STEERING LEFT and the rule is right hand y axis position is MIDDLE, right hand x axis position is INSIDE, right hand has NO MOTION, left hand y axis position is MIDDLE, left hand has MOTION on y axis, left hand has NO MOTION on x axis.

35. The method of claim 9 wherein the gesture is LAUNCH BAR UP and the rule is a sequence of two distinct positions, 1) both hands x axis position is INSIDE, right hand y axis position is MIDDLE, left hand y axis position is VERY LOW, left hand is EXTENDED, both hands have NO MOTION, then, within about one second, 2) right hand y axis position is MIDDLE, both hands x axis position is INSIDE, left hand y axis position is HIGH, both hands have NO MOTION.

36. The method of claim 9 wherein the gesture is LAUNCH BAR DOWN and the rule is a sequence of two distinct positions, 1) right hand y axis position is MIDDLE, both hands x axis position is INSIDE, left hand y axis position is HIGH, both hands have NO MOTION, then, within about one second, 2) both hands x axis position is INSIDE, right hand y axis position is MIDDLE, left hand y axis position is VERY LOW, left hand is EXTENDED, both hands NO MOTION.

37. The method of claim 9 wherein the gesture is SPREAD WINGS/HELICOPER BLADES and the rule is a sequence of two static poses:

1) both hands y axis position is MIDDLE, both hands x axis position is INSIDE, both hands have NO MOTION, then, within about one second

     2) both hands y axis position is MIDDLE, both hands are EXTENDED, both hands have NO MOTION, both hands x axis position is OUTSIDE.

38. The method of claim 9 wherein the gesture is UP HOOK and the rule is a sequence of two static poses:

1) right hand y axis position is MIDDLE, right hand x axis position is INSIDE, right hand has NO MOTION, left hand x axis position is INSIDE, left hand has NO MOTION, left hand is EXTENDED, left hand y axis position is VERY LOW, then, within about one second,

     2) both hands y axis position is MIDDLE, both hands x axis position is INSIDE, both hands NO MOTION.

39. The method of claim 9 wherein the gesture is DOWN HOOK and the rule is a sequence of two static poses:

1) right hand y axis position is MIDDLE, right hand x axis position is INSIDE, right hand has NO MOTION, left hand x axis position is INSIDE, left hand has NO MOTION, left hand y axis position is VERY HIGH or HIGH, then within about one second, the next pose is

     2) both hands y axis position is MIDDLE, both hands x axis position is INSIDE, both hands have NO MOTION.

40. The method of claim 9 wherein the gesture is FOLD WINGS/HELICOPTER BLADES and the rule is a sequence of two static poses:

1) both hands y axis position is MIDDLE, both hands are EXTENDED, both hands have NO MOTION, both hands x axis position is OUTSIDE, then within about one second

     2) both hands y axis position is MIDDLE, both hands x axis position is INSIDE, both hands have NO MOTION.

41. A computer readable medium containing a computer program for performing a method of identifying a human gesture, the method comprising transforming a time sequence of data related to the human gesture into waveforms;

extracting features from the waveforms; and

identifying the human gesture based on the extracted features.

42. An apparatus for identifying a human gesture, comprising:

means for providing a time sequence of data related to the human gesture;

means for transforming the time sequence of data into waveforms;

means for extracting features from the waveforms; and

means for identifying the human gesture based on the extracted features.

43. The apparatus of claim 42 wherein the means for providing comprises accelerometers.

44. The apparatus of claim 42 wherein the means for providing comprises at least one video camera.

45. The apparatus of claim 42 wherein the means for transforming, the means for extracting and the means for identifying comprise at least one computer.