System and method for selecting features for identifying human activities in a human-computer interacting environment

• US 9,536,145 B2
• Filed: 12/19/2014
• Issued: 01/03/2017
• Est. Priority Date: 12/27/2013
• Status: Active Grant

• Current Assignee: TATA Consultancy Services Limited (Tata Sons Pvt Ltd.)
• Original Assignee: TATA Consultancy Services Limited (Tata Sons Pvt Ltd.)
• Inventors: Vempada, Ramu Reddy, Chattopadhyay, Tanushyam
• Primary Examiner(s): Heidemann, Jason
• Application Number:
  US14/576,627
• Publication Number:
  US 20150186724A1
• Time in Patent Office: 746 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: A61B 5/1123 Discriminating type of movement, e.g. walking or running A61B5/1116, A61B5/112 take precedence
• CPC Class Codes: A61B 5/1128 using image analysis A61B5/1127 takes precedence
• CPC Class Codes: A61B 5/7267 involving training the classification device
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 10/34 Smoothing or thinning of the pattern; Morphological operations; Skeletonisation
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: G06V 40/25 Recognition of walking or running movements, e.g. gait recognition
• CPC Class Codes: G16H 50/70 for mining of medical data, e.g. analysing previous cases of other patients

Abstract

A System and method for identifying one or more human activities in a human-computer interacting environment. Skeleton points associated with a human are received. A data variation factor for the skeleton points is calculated, and a set of skeleton points is selected based on the data variation factor. One or more features are defined from the set of skeleton points by identifying a variance in coordinates of the set of skeleton points by using one or more statistical parameters. The one or more features are used to identify the one or more human activities.

Claims

1. A system for selecting one or more features to identify one or more human activities in a human-computer interacting environment, the system comprising:

a processor;

a non-transitory memory coupled to the processor, wherein the processor is capable of executing a plurality of modules stored in the memory, and wherein the plurality of modules comprise;

a receiving module that receives skeleton points associated with one or more humans, wherein the one or more humans perform the one or more human activities, and wherein the skeleton points comprise a plurality of position coordinates of the one or more humans;

a computation module that;

calculates a data variation factor for the skeleton points, wherein the data variation factor identifies a variation between two or more of the plurality of position coordinates of the one or more human, and wherein the data variation factor is indicative of a variation between the one or more humans performing one or more activities and a variation between the one or more human activities;

scales the data variation factor with respect to a maximum and a minimum value and sorts the data variation factor by distance metrics in a descending order such that the skeleton point is independent of the one or more human; and

computes the distance metrics by scaling first four maximum values, wherein first four maximum values correspond to five position coordinates from amongst the plurality of position coordinates;

a selection module that selects a set of skeleton points from the skeleton points based on the scaled data variation factor;

a feature defining module that;

identifies a change in position coordinates associated with the set of skeleton points by using one or more statistical parameters, wherein the set of skeleton points defines the one or more human activity to be identified; and

extracts one or more features from the set of skeleton points based on the change in the position coordinates; and

an identification module to identify the one or more human activities based on the extracted one or more features.

2. The system of claim 1, wherein the skeleton points comprise 20 skeleton joints, wherein the 20 skeleton joints comprise a Hip Center, a Spine, a Shoulder Center, a Head, a Shoulder Left, an Elbow Left, a Wrist Left, a Hand Left, a Shoulder Right, an Elbow Right, a Wrist Right, a Hand Right, a Hip Left, a Knee Left, an Ankle Left, a Foot Left, a Hip Right, a Knee Right, an Ankle Right, and a Foot Right.

3. The system of claim 1, wherein the one or more human activities comprises standing, sitting, walking, and lying.

4. The system of claim 1, wherein the selection module selects data variation factors close to a particular value of the distance metrics, wherein the particular value includes a value equal to 1.

5. The system of claim 1, wherein the computation module calculates the data variation factor based on a standard deviation value for the skeleton points.

6. The system of claim 1, wherein the set of skeleton points comprise a Head, a Shoulder Center, a Shoulder Left, and a Shoulder Right.

7. The system of claim 1, wherein the identification module executes a Support Vector Machine (SVM) learning algorithm over the one or more features to identify the one or more human activities.

8. The system of claim 1, wherein the one or more statistical parameters comprise a mean, a standard deviation, and a maximum and a minimum position coordinates of the set of skeleton points associated with the one or more human activities.

9. The system of claim 1, wherein the variation between two or more of the plurality of position coordinates of the set of skeleton points is identified for a period of 30 frames.

10. The system of claim 1, wherein the one or more features comprises a mean of the set of skeleton points and a difference between a maximum and a minimum of the plurality of position coordinates associated with the one or more human activities.

11. A method for selecting one or more features to identify one or more human activities in a human-computer interacting environment, the method comprising:

receiving skeleton points associated with one or more humans, wherein the one or more humans performs the one or more human activities, and wherein the skeleton points comprise position coordinates of the one or more human;

calculating a data variation factor for the skeleton points, wherein the data variation factor identifies a variation between the position coordinates of the one or more humans and wherein the data variation factor is indicative of a variation between the one or more humans performing one or more activities and a variation between the one or more human activities;

scaling the data variation factor with respect to a maximum and a minimum value and sorts the data variation factor by distance metrics in a descending order such that the skeleton point is independent of the one or more human;

computing the distance metrics by scaling first four maximum values, wherein first four maximum values correspond to five position coordinates from amongst the plurality of position coordinates;

selecting a set of skeleton points from the skeleton points based on the scaled data variation factor;

identifying a change in position coordinates associated with the set of skeleton points by using one or more statistical parameters, wherein the set of skeleton points defines one or more human activity to be identified;

extracting one or more features from the set of skeleton points based on the change in the position coordinates; and

identifying the one or more human activities by using the extracted one or more features.

12. The method of claim 11, wherein the one or more human activities is identified by executing a Support Vector Machine (SVM) learning algorithm over the one or more features.

13. A non-transitory computer storage medium having embodied thereon a computer program for selecting one or more features to identify one or more human activities in a human-computer interacting environment, the method comprising:

a program code for receiving skeleton points associated with one or more humans, wherein the one or more humans performs the one or more human activities, and wherein the skeleton points comprise position coordinates of the one or more human;

a program code for calculating a data variation factor for the skeleton points, wherein the data variation factor identifies a variation between the position coordinates of one or more humans, and wherein the data variation factor is indicative of a variation between the one or more humans performing one or more activities and a variation between the one or more human activities;

a program code for scaling the data variation factor with respect to a maximum and a minimum value and sorts the data variation factor by distance metrics in a descending order such that the skeleton point is independent of the one or more human;

a program code for computing the distance metrics by scaling first four maximum values, wherein first four maximum values correspond to five position coordinates from amongst the plurality of position coordinates;

a program code for selecting a set of skeleton points from the skeleton points based on the scaled data variation factor;

a program code for identifying a change in the position coordinates associated with the set of skeleton points by using one or more statistical parameters, wherein the set of skeleton points defines the one or more human activities to be identified;

a program code for extracting one or more features from the set of skeleton points based on the change in the position coordinates; and

a program code for identifying the one or more human activities by using the extracted one or more features.