Identification of a gesture

• US 9,361,512 B2
• Filed: 07/08/2013
• Issued: 06/07/2016
• Est. Priority Date: 08/03/2012
• Status: Active Grant

• Current Assignee: Crunchfish Gesture Interaction AB
• Original Assignee: Crunchfish AB
• Inventors: Cronholm, Paul
• Primary Examiner(s): AZARIAN, SEYED H
• Application Number:
  US14/419,098
• Publication Number:
  US 20150220776A1
• Time in Patent Office: 1,065 Days
• Field of Search: 382/100, 382/103, 382106-107, 382/155, 382/162, 382/168, 382/173, 382/181, 382/189, 382/199, 382/209, 382/219, 382/224, 382/232, 382/254, 382/274, 382/276, 382/286, 382/291, 382/305, 382/312, 345/173, 348/333.01
• US Class Current: 1/1
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

A computing device for identifying a gesture performed by a tracked object in an image stream, said computing device comprising a memory and a controller, wherein said controller is configured to: determine a movement vector for the tracked object, compare the movement vector against a reference library of gestures, and thereby identify a gesture matching the movement vector, wherein said movement vector comprises at least one partial movement having a direction being identified by a relative angle and wherein said comparison is based on said relative angle.

Claims

1. A computing device for identifying a gesture performed by a tracked object in an image stream, said computing device comprising a memory and a controller, wherein said controller is configured to:

determine a movement vector for the tracked object;

compare the movement vector against a reference library of gestures; and

identify a gesture matching the movement vector based on the comparison, wherein

said movement vector comprises at least one partial movement having a direction being identified by an angle and wherein said comparison is based on said angle, and

said controller is further configured to

correct for a misalignment of said computing device by determining said angle as a relative angle between a direction of a first partial movement and a direction of a second partial movement, and

traverse a state diagram, wherein a transition corresponds to an element of said movement vector and at least one state corresponds to an action;

whereby the controller executes a transition from a state to another according to elements of the movement vector until a gesture is identified by reaching the corresponding state.

2. The computing device of claim 1, wherein said controller is further configured to determine a base direction for adapting said relative angle or for adapting a library angle to be compared with.

3. The computing device of claim 2, wherein said controller is further configured to determine said base direction by comparing a structure of the tracked object with a model structure.

4. The computing device of claim 1, wherein the state diagram, is a Turing machine.

5. The computing device of claim 4, wherein said state diagram comprises at least one stable state, wherein a stable state is a state corresponding to an action or an initial state and

wherein said controller is further configured to return to a previous stable state when a no other transition exists in the state diagram for the elements in the movement vector.

6. The computing device of claim 5, wherein said controller is further configured to return to an initial state of said state diagram if no further partial movement is detected within a timeout threshold.

7. The computing device of claim 4, wherein said controller is further configured to preemptively execute an action corresponding to an assumed gesture as said gesture is being detected, wherein the controller is configured to assume that the gesture being detected is an assumed gesture if the state of the state diagram currently being visited is on a transition path to said gesture to be detected.

8. The computing device of claim 7, wherein said controller is further configured to present or effect a result of the actions corresponding to the assumed gesture as the state for the gesture to be detected is reached.

9. The computing device of claim 4, wherein said state transition diagram comprises at least one stable state on a path to another stable state for enabling a first gesture to be defined by a sub sequence of movements for a second gesture.

10. The computing device of claim 4, wherein said controller is further configured to identify a gesture for a plurality of objects by traversing a plurality of state diagrams each being associated with each of the plurality of objects, wherein said controller is further configured to identify a gesture for said plurality of objects being tracked by traversing the associated state diagram, wherein a transition in an associated state diagram corresponds to an element of a movement vector for the associated object to be tracked.

11. The computing device of claim 4, wherein said controller is further configured to identify a gesture for a plurality of objects by traversing a state diagram wherein each transition corresponds to a movement of the plurality of objects.

12. The computing device of claim 1, wherein said controller is further configured to determine that an angle matches a library angle to be compared with if the angle being compared is within an error tolerance of said library angle.

13. The computing device of claim 1, wherein said controller is further configured to:

determine a distance of a movement;

determine if the distance of the movement is above a first distance threshold value; and

if so,

compare the angle of the movement with the library angle.

14. The computing device of claim 13, wherein said controller is configured to differentiate between a fast and a slow movement.

15. The computing device of claim 13, wherein said controller is configured to differentiate between a short and a long movement.

16. The computing device of claim 1, wherein said movement vector comprises at least one further dimension taken from a group consisting of:

depth, rotation, degrees of turning, distance of movement and speed of movement.

17. The computing device of claim 1, wherein said controller is configured to determine a distance based on a size of the object to be tracked, a distance to the object to be tracked, size of a display and the field of view of a camera.

18. A method for use in a computing device for identifying a gesture performed by a tracked object in an image stream, said method comprising:

determining a movement vector for the tracked object;

comparing the movement vector against a reference library of gestures; and

identifying a gesture matching the movement vector based on the comparison, wherein

said movement vector comprises at least one partial movement having a direction being identified by an angle and wherein said comparison is based on said angle, and

said method further comprises

correcting for a misalignment of said computing device by determining said angle as a relative angle between a direction of a first partial movement and a direction of a second partial movement, and

traversing a state diagram, wherein a transition corresponds to an element of said movement vector and at least one state corresponds to an action;

whereby the controller executes a transition from a state to another according to elements of the movement vector until a gesture is identified by reaching the corresponding state.

19. A non-transitory computer readable storage medium encoded with instructions that, when loaded and executed on a processor, causes the method according to claim 18 to be performed.