Method and apparatus for performing scale-invariant gesture recognition

• US 6,944,315 B1
• Filed: 10/31/2000
• Issued: 09/13/2005
• Est. Priority Date: 10/31/2000
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Zipperer, John B., Wood, Stephen V., Martins, Fernando C. M.
• Primary Examiner(s): Patel, Kanjibhai
• Assistant Examiner(s): Azarian, Seyed
• Application Number:
  US09/704,385
• Time in Patent Office: 1,778 Days
• Field of Search: 348/94, 348/97, 348/169, 348/413.1, 348/113, 348/116, 348/734, 348/14.15, 382/103, 382/104, 382/106, 382/107, 382/118, 382/123, 382/201, 382/206, 382/209, 382/225, 382/232, 382/236, 382/253, 382/268, 382/284, 382/291, 382/296, 382/305, 382/224, 382/235, 382/286, 382/295, 382/317, 382/321, 345/156, 345/159, 345/167, 345/419, 345/863, 342/457, 370/395.63
• US Class Current: 382/103
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training

Abstract

A gesture recognition process includes tracking an object in two frames of video, determining differences between a location of the object in one frame of the video and a location of the object in another frame of the video, obtaining a direction of motion of the object based on the differences, and recognizing a gesture of the object based, at least in part, on the direction of motion of the object.

Claims

1. A method comprising:

tracking an object in two frames of video;

determining differences between a location of the object in one frame of the video and a location of the object in another frame of the video;

obtaining a direction of motion of the object based on the differences; and

recognizing a gesture performed by the object based, at least in part, on the direction of motion of the object;

wherein the differences comprise first order differences between coordinate locations of the object in the two frames of video.

2. The method of claim 1, wherein:

the coordinate locations of the object comprise Cartesian X and Y coordinate locations; and

the first order differences comprise a difference in the X coordinate location of the object in the two frames of video and a difference in the Y coordinate location of the object in the two frames of video.

3. The method of claim 1, wherein the direction of motion corresponds to a value that is obtained based on a quotient of the differences.

4. The method of claim 3, wherein the value is obtained by taking an inverse tangent of the quotient of the differences.

5. The method of claim 1, further comprising:

repeating, N times (N

     1), tracking, determining and obtaining to obtain N vectors, each of the N vectors defining a direction of motion of the object, the N vectors together defining a gesture performed by the object;

wherein recognizing is performed using the N vectors.

6. The method of claim 1, further comprising:

selecting one of predefined gestures that most closely matches the gesture of the object.

7. The method of claim 1, wherein tracking comprises:

capturing the two frames of video between indicators;

obtaining the location of the object in the one frame of video; and

obtaining the location of the object in the other frame of video.

8. The method of claim 7, wherein the indicators comprise perceptible indicators.

9. The method of claim 1, wherein the direction of motion and gesture recognition is invariant relative to a scale and a translation of the object.

10. An article comprising:

a computer-readable medium that stores computer-executable instructions that cause a computer to;

track an object in two frames of video;

determine differences between a location of the object in one frame of the video and a location of the object in another frame of the video;

obtain a direction of motion of the object based on the differences; and

recognize a gesture performed by the object based, at least in part, on the direction of motion of the object;

wherein the differences comprise first order differences between coordinate locations of the object in the two frames of video.

11. The article of claim 10, wherein:

the coordinate locations of the object comprise Cartesian X and Y coordinate locations; and

the first order differences comprise a difference in the X coordinate location of the object in the two frames of video and a difference in the Y coordinate location of the object in the two frames of video.

12. The article of claim 10, wherein the direction of motion corresponds to a value that is obtained based on a quotient of the differences.

13. The article of claim 12, wherein the value is obtained by taking an inverse tangent of the quotient of the differences.

14. The article of claim 10, further comprising instructions that cause the computer to:

repeat, N times (N

     1), tracking, determining and obtaining to obtain N vectors, each of the N vectors defining a direction of motion of the object, the N vectors together defining a gesture performed by the object;

wherein recognizing is performed using the N vectors.

15. The article of claim 10, further comprising instructions that cause the computer to:

select one of predefined gestures that most closely matches the gesture of the object.

16. The article of claim 10, wherein tracking comprises:

capturing the two frames of video between indicators;

obtaining the location of the object in the one frame of video; and

obtaining the location of the object in the other frame of video.

17. The article of claim 16, wherein the indicators comprise perceptible indicators.

18. The article of claim 10, wherein the direction of motion and gesture recognition is invariant relative to a scale and a translation of the object.

19. An apparatus for performing gesture recognition, comprising:

a camera to capture a video of an object, the object performing a gesture;

a feature extraction mechanism to track the object in frames of the video and to extract a feature sequence based on the frames, the feature sequence including a direction of motion of the object obtained based on differences between a location of the object in one frame of the video and a location of the object in another frame of the video; and

a gesture recognition mechanism to recognize the gesture performed by the object based, at least in part, on the extracted feature sequence;

wherein the differences comprise first order differences between coordinate locations of the object in the two frames of the video.

20. The apparatus of claim 19, wherein:

the coordinate locations of the object comprise Cartesian X and Y coordinate locations; and

the first order differences comprise a difference in the X coordinate location of the object in the two frames of video and a difference in the Y coordinate location of the object in the two frames of video.

21. The apparatus of claim 19, wherein the direction of motion corresponds to a value that is obtained based on a quotient of the differences.

22. The apparatus of claim 21, wherein the value is obtained by taking an inverse tangent of the quotient of the differences.

23. The apparatus of claim 19, wherein the processor executes instructions to:

select one of predefined gestures that most closely matches the gesture of the object.

24. The apparatus of claim 19, wherein the feature extraction mechanism comprises:

an object tracker component to capture frames of the video between perceptible indicators and to obtain a location of the objection in a frame of the video; and

a direction extraction block to obtain the direction of motion of the object based on the frames and locations of the objection in the frames of the video.

25. The apparatus of claim 19, wherein the direction of motion and gesture recognition is invariant relative to a scale and a translation of the object.

26. The apparatus of claim 19, wherein the gesture recognition mechanism comprises:

a plurality of predefined gesture units to store a plurality of predefined gestures, and to produce a probability of the gesture matching a predefined gesture, based on the feature sequence extracted by the feature extraction mechanism; and

a comparator to select one of predefined gestures that most closely matches the gesture of the object, based on probabilities obtained by the plurality of predefined gesture units.