SYSTEMS AND METHODS OF DETECTING BODY MOVEMENTS USING GLOBALLY GENERATED MULTI-DIMENSIONAL GESTURE DATA

• US 20140198954A1
• Filed: 04/18/2012
• Published: 07/17/2014
• Est. Priority Date: 07/28/2011
• Status: Active Grant

• Current Assignee: ARB Labs Inc.
• Original Assignee: Adrian Bulzacki
• Inventors: Bulzacki, Adrian
• Granted Patent: US 9,639,746 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06V 10/75 Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: Y02A 90/10 Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Abstract

The disclosure describes systems and methods of detecting body movements using gesture data. The gesture data may be self-referenced and may be comprised by frames which may identify locations or positions of body parts of a subject with respect to a particular reference point within the frame. A classifier may process frames to learn body movements and store the frames of gesture data in a database. Data comprising frames of self-referenced gesture data may be received by a recognizer which recognizes movements of the subject identified by the frames by matching gesture data of the incoming frames to the classified self-referenced gesture data stored in the database.

Claims

1. A method of identifying a movement of a subject based on data, the method comprising:

receiving, by a system via a network, a frame comprising gesture data identifying positions of one or more body parts of a subject with respect to waist of the subject'"'"'s body;

assigning, by a classifier of the system, the frame to a first movement;

receiving, by a recognizer of the system, a new gesture data identifying positions of one or more body parts of a new subject with respect to waist of the new subject'"'"'s body;

determining, by the recognizer, that movement of the new subject corresponds to the first movement responsive to comparing at least a portion of the new gesture data to at least a portion of the gesture data of the frame.

2. The method of claim 1, further comprising identifying, by the recognizer, within a first threshold of accuracy that the at least the portion of the new gesture data matches the at least the portion of the gesture data, and

further determines that the movement of the new subject is the first movement based on the identification within the first threshold.

3. The method of claim 2, further comprising:

additionally identifying, by the recognizer, within a second threshold of greater certainty than the first threshold of certainty that at least a second portion of the new gesture data matches at least a second portion of the gesture data, and

further determines with a greater certainty that the movement of the new subject corresponds to the first movement based on the identification within the second threshold.

4. The method of claim 1, further comprising determining that the movement of the new subject corresponds to the first movement responsive to comparing one or more features of the gesture data of the frame to the one or more features of the new gesture data.

5. The method of claim 1, further comprising storing, by the system, the frame in a database, and determining that the movement of the new subject corresponds to the first movement upon comparing the one or more features of the new gesture data to the one or more features of the frame in the database.

6. The method of claim 1, further comprising:

receiving a plurality of frames comprising gesture data via the network from a plurality of users at different geographical locations,

receiving the frame via the network from a user of the plurality of users,

storing the gesture data comprising the frame in a database, and

retrieving the frame from the database upon detecting that gesture data in the frame substantially matches the new gesture data.

7. The method of claim 1, further comprising comparing, by the recognizer, a feature of the new gesture data identifying a position of a shoulder of the new subject with respect to the new subject'"'"'s waist to a feature of the gesture data in the frame identifying the position of a shoulder of the subject with respect to the subject'"'"'s waist.

8. The method of claim 1, further comprising comparing, by the recognizer, a feature of the new gesture data identifying a position of a hip of the new subject with respect to the new subject'"'"'s waist to a feature of the gesture data in the frame identifying the position of a hip of the subject with respect to the subject'"'"'s waist.

9. The method of claim 1, further comprising comparing, by the recognizer, a feature of the new gesture data identifying a position of an elbow of the new subject with respect to the new subject'"'"'s waist to a feature of the gesture data in the frame identifying the position of an elbow of the subject with respect to the subject'"'"'s waist.

10. The method of claim 1, further comprising comparing, by the recognizer, a feature of the new gesture data identifying a position of a palm of the new subject with respect to the new subject'"'"'s waist to a feature of the gesture data in the frame identifying the position of a palm of the subject with respect to the subject'"'"'s waist.

11. The method of claim 1, further comprising comparing, by the recognizer, a feature of the new gesture data identifying a position of one or more fingers of the new subject with respect to the new subject'"'"'s waist to a feature of the gesture data in the frame identifying the position of one or more fingers of the subject with respect to the subject'"'"'s waist.

12. The method of claim 1, further comprising comparing, by the recognizer, a feature of the new gesture data identifying a position of a knee of the new subject with respect to the new subject'"'"'s waist to a feature of the gesture data in the frame identifying the position of a knee of the subject with respect to the subject'"'"'s waist.

13. The method of claim 1, further comprising comparing, by the recognizer, a feature of the new gesture data identifying a position of a heel of the new subject with respect to the new subject'"'"'s waist to a feature of the gesture data identifying the position of a heel of the subject with respect to the subject'"'"'s waist.

14. The method of claim 1, further comprising comparing, by the recognizer, a feature of the new gesture data identifying a position of toes of the new subject with respect to the new subject'"'"'s waist to a feature of the gesture data identifying the position of toes of the subject with respect to the subject'"'"'s waist.

15. The method of claim 1, further comprising comparing, by the recognizer, a feature of the new gesture data identifying a position a portion of the head of the new subject with respect to the new subject'"'"'s waist to a feature of the gesture data in the frame identifying the position of a portion of the head of the subject with respect to the subject'"'"'s waist.

16. The method of claim 1, further comprising comparing, by the recognizer, a feature of the new gesture data identifying a position of the neck of the new subject with respect to the new subject'"'"'s waist to a feature of the gesture data in the frame identifying a position of the neck of the subject with respect to the subject'"'"'s waist.

17. The method of claim 1, further comprising comparing, by the recognizer, a feature of the new gesture data identifying a position of a pelvis of the new subject with respect to the new subject'"'"'s waist to a feature of the gesture data in the Atone identifying the position of a pelvis of the subject with respect to the subject'"'"'s waist.

18. The method of claim 1, further comprising comparing, by the recognizer, a feature of the new gesture data identifying a position of a belly of the new subject with respect to the new subject'"'"'s waist to a feature of the gesture data in the frame identifying the position of a belly of the subject with respect to the subject'"'"'s waist.

19. The method of claim 1, wherein the positions of one or more body parts are expressed as one or more distances between the one or more body parts of the subject and the waist of the subject and wherein the values indicating the one or more distances are scaled and normalized to values between 0 and 1.

20. The method of claim 1, further comprising receiving, by the system from a detector, the frame comprising gesture data, the detector comprising a camera comprising a functionality to extrapolate self-referential gesture data, and

receiving, by the recognizer, from a different detector the new gesture data, the different detector comprising a different camera that comprises the functionality to extrapolate self-referential gesture data.

21. A system for identifying a movement of a subject based on data, the system comprising:

a database storing a frame received via a network, the frame comprising gesture data identifying positions of one or more body parts of a subject with respect to waist of the subject'"'"'s body;

a classifier assigning the frame to a first movement;

a recognizer receiving a new gesture data identifying positions of one or more body parts of a new subject with respect to waist of the new subject'"'"'s body, wherein the recognizer determines that a movement of the new subject corresponds to the first movement responsive to comparing at least a portion, of the new gesture data to the at least a portion of the gesture data in the frame stored in the database.

22. The system of claim 21, wherein the recognizer determines within a first threshold of certainty that the movement of the new subject corresponds to the first movement.

23. The system of claim 22, further comprising:

the recognizer additionally determining within a second threshold of greater certainty than the first threshold, that the movement of the new subject corresponds to the first movement responsive to comparing the new gesture data to a second frame assigned to the first movement.

24. The system of claim 21, wherein the system determines that the movement of the new subject corresponds to the first movement responsive to comparing the one or more positions of the frame to the one or more positions of the new gesture data.

25. The system of claim 21, wherein the system determines that the movement of the new subject corresponds to the first movement upon comparing the one or more positions identified by the new gesture data to the one or more positions identified by the gesture data in the frame.

26. The system of claim 21, wherein a plurality of frames comprising the gesture data are received via the network from a plurality of users at different geographical locations, and wherein the frame is received via the network from a user of the plurality of users.

27. The system of claim 21, wherein the system receives from a detector the frame comprising gesture data, the detector comprising a camera comprising a functionality to extrapolate self-referential gesture data, and wherein the recognizer receives from a different detector the new gesture data, the different detector comprising a different camera that comprises the functionality to extrapolate self-referential gesture data.

28. The system of claim 27, wherein the camera is a Microsoft Kinect Camera.