SYSTEMS AND METHODS OF IDENTIFYING A GESTURE USING GESTURE DATA COMPRESSED BY PRINCIPAL JOINT VARIABLE ANALYSIS

• US 20130278501A1
• Filed: 03/15/2013
• Published: 10/24/2013
• Est. Priority Date: 04/18/2012
• Status: Active Grant

• Current Assignee: ARB Labs Inc.
• Original Assignee: ARB Labs Inc.
• Inventors: BULZACKI, Adrian
• Granted Patent: US 9,448,636 B2
• Time in Patent Office: Days
• US Class Current: 345/157
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/847 Cooperative playing, e.g. requiring coordinated actions from several players to achieve a common goal
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: G06F 18/2135 based on approximation criteria, e.g. principal component analysis
• CPC Class Codes: G06F 18/2413 based on distances to training or reference patterns
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06V 10/7715 Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

Systems and method described herein present techniques for identifying a gesture using gesture data compressed by principal joint variable analysis. A classifier of a gesture recognition system may receive a frame comprising a set of gesture data points identifying locations of body parts of a subject. The classifier may determining that a subset of the set of gesture data points is sufficient to recognize a first gesture. The subset may be stored into a database in reference to the first gesture. A recognizer may receive a new frame of new gesture data points identifying locations of body parts of a new subject. The recognizer may recognize that the gesture of the new subject corresponds to the first gesture responsive to comparing at least one new gesture data point from the new frame to at least one gesture data point of the subset.

Claims

1. ) A method of identifying a gesture of a subject based on self-referenced gesture data compressed by principal joint variable analysis, the method comprising:

receiving, by a classifier of a system, a frame comprising a set of gesture data points, each gesture data point identifying a location of a part of a body of a subject with respect to a reference point on the body of the subject;

determining, by the classifier, that a subset of the set of gesture data points is sufficient to recognize a first gesture;

storing, into a database of the system, the subset in reference to the first gesture;

receiving, by a recognizer of the system, a new frame of new gesture data points, each new gesture data point identifying a location of a part of a body of a new subject with respect to a reference point on the body of the new subject; and

recognizing, by the recognizer, that the gesture of the new subject corresponds to the first gesture responsive to comparing at least one new gesture data point from the new frame to at least one gesture data point of the subset.

2. ) The method of claim 1, further comprising determining, by the system, that the new subject performed the first gesture in response to the recognition.

3. ) The method of claim 1, further comprising selecting, by the recognizer, new gesture data points within a subset of the new gesture data points for identifying the first gesture, the new gesture data points in the new subset corresponding to the gesture data points of the subset.

4. ) The method of claim 1, further identifying, by the recognizer, within a first threshold of accuracy that at least a first new gesture data point of the frame matches at least one gesture data point of the subset.

5. ) The method of claim 4, further identifying, by the recognizer, within a second threshold of greater accuracy than the first threshold of accuracy that at least a second new gesture data point matches at least a second gesture data point of the subset.

6. ) The method of claim 1, further comprising:

receiving, by the classifier, together with the frame, a second frame comprising a second set of gesture data points, the second set of gesture data points having gesture data values different from the data values of the set of gesture data points, and

extracting a second subset of the gesture data points, the second subset comprising the same gesture data points as the subset.

7. ) The method of claim 1, wherein the reference point includes a waist of the body of the subject.

8. ) The method of claim 1, wherein the reference point includes the mid-point of the chest of the subject.

9. ) The method of claim 1, further comprising comparing, by the recognizer, a new gesture data point corresponding to a location of a wrist of the new subject with respect to the new subject'"'"'s waist to a gesture data point of the wrist of the subject with respect to the subject'"'"'s waist.

10. ) The method of claim 1, further comprising comparing, by the recognizer, a new gesture data point corresponding to a location of an elbow of the new subject with respect to the new subject'"'"'s mid-point chest to a gesture data point corresponding to a location of an elbow of the subject with respect to the subject'"'"'s waist.

11. ) A system for using self-referenced gesture data compressed by principal joint variable analysis to identify a gesture of a subject, the system comprising:

a classifier of a system receiving a frame comprising a set of gesture data points, each gesture data point identifying a location of a part of a body of a subject with respect to a reference point on the body of the subject and determining that a subset of the set of gesture data points is sufficient to recognize a first gesture;

the system storing into a database of the system the subset in reference to the first gesture; and

a recognizer of the system receiving a new frame of new gesture data points, each new gesture data point identifying a location of a part of a body of a new subject with respect to a reference point on the body of the new subject and recognizing that the gesture of the new subject corresponds to the first gesture responsive to comparing at least one new gesture data point from the new frame to at least one gesture data point of the subset.

12. ) The system of claim 11, wherein the system further determines that the new subject performed the first gesture in response to the recognition.

13. ) The system of claim 11, wherein the recognizer further selects the new gesture data points within a subset of the new gesture data points for identifying the first gesture, the new gesture data points in the new subset corresponding to the gesture data points of the subset.

14. ) The system of claim 11, wherein the recognizer further identifies, within a first threshold of accuracy, that at least a first new gesture data point of the frame matches at least one gesture data point of the subset.

15. ) The system of claim 14, wherein the recognizer further identifies, within a second threshold of greater accuracy than the first threshold of accuracy, that at least a second new gesture data point matches at least a second gesture data point of the subset.

16. ) The system of claim 11, wherein the classifier further receives, together with the frame, a second frame comprising a second set of gesture data points, the second set of gesture data points having gesture data values different from the data values of the set of gesture data points, and extracts a second subset of the gesture data points, the second subset comprising the same gesture data points as the subset.

17. ) The system of claim 11, wherein the reference point includes a waist of the body of the subject.

18. ) The system of claim 11, wherein the reference point includes the mid-point of the chest of the subject.

19. ) The system of claim 11, wherein the recognizer compares a new gesture data point corresponding to a location of a wrist of the new subject with respect to the new subject'"'"'s waist to a gesture data point of the wrist of the subject with respect to the subject'"'"'s waist.

20. ) The system of claim 11, wherein the recognizer compares a new gesture data point corresponding to a location of an elbow of the new subject with respect to the new subject'"'"'s mid-point chest to a gesture data point corresponding to a location of an elbow of the subject with respect to the subject'"'"'s waist.